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Dear Friends and Colleagues, 



This volume contains almost all of the presentations given at what was an outstanding 
23rd Annual Conference of the Northeast Association for Institutional Research, held 
in Princeton, New Jersey. Attendance was high, the weather good, the location ideal, 
and best of all, the quality of the sessions was consistently high. I especially 
recommend to you the paper by Robert Toutkoushian which won the Best Paper 
Award for this conference. It is unfortunate that other highly regarded elements of the 
conference, such as the Newcomers Workshop, the two Statistics Workshops, the 
Web Basics Workshop, and the final panel presentation, cannot also be included here. 

The list of people deserving thanks for the success of the conference is about as long 
as the list of attendees. Barbara Palmer deserves an award for the Best Program Put 
Together By A Person Who Had No Chips to Call In; Eleanor Swanson for 
Outstanding Local Arrangements Organization by a Self-Styled Disorganized Person; 
and, as usual, Brenda Bretz wins both the Congeniality and Talent awards, as well as 
the running-suit competition. I would also like to thank Corby Coperthwaite 
(Publications Chair), Marian Pagano (Mentor program). Marge Wiseman (conference 
evaluation), Denise Krallman (1997Program Chair), Bob Yanckello (1997 Local 
Arrangements Chair), and Sandy Johnson, Tony Broh, and other NJAIR members for 
their help. Finally, of course, all of us who worked on the Princeton conference for 
more than a year must express our appreciation to all of you who sent in proposals, 
made your presentations, and served as both appreciative and critical audiences. 
Despite all the hard work of those named above, there would have been no conference 
at all without you. 

The Proceedings that you are holding, by the way, may soon become a collectors 
item: Jennifer Brown, 1996-97 NEAIR President, has formed a committee to 
investigate the development of an NEAIR Website, which might eventually include 
our conference Proceedings, although hard copies would be available on request. 

Stay tuned for further developments! 

I wish you the best of health, wealth, and enrollments for 1997, and look forward to 
seeing you in Hartford. 



Ellen A. Kanarek 
President, NEAIR 1995-96 
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USING COHORT ANALYSIS TO EVALUATE THE 
IMPACT OF A SUPPORT PROGRAM FOR MINORITY STUDENTS 

Hershel Alexander 
Research and Planning Analyst 
Prince George's Community College 



Introduction 

Over the years, Prince George's Community College has demonstrated 
commitment to its growing proportion of minority students by tracking minority 
student progress and by establishing support programs. This paper illustrates how the 
college's Office of Institutional Research and Analysis evaluated one of these 
programs. Following an overview of the methodology and preceding a summary of 
the report, findings from this program evaluation are highlighted in 5 areas: 
Developmental Needs and Course Loads; Retention Rates; Cumulative Credit Hours; 
Average GPAs; and the Prince George's outcomes typology. 

The research office's experience with college-wide cohort studies facilitated 
the acceptance of its first application of a cohort methodology to a program 
evaluation. But to which program? Since minority students had increased to about 3 
quarters of all first-time freshmen and since these minority students had been 
achieving at lower levels than other first-time freshmen, the college decided upon the 
only support program for minority students exclusively: ALANA, a program for 
African, Latin, Asian, and Native American students that offered services such as 
college orientation sessions, cultural field trips, faculty or staff mentors, 
motivational/informational newsletters, social receptions, and referrals to other areas 
for additional assistance. Originally, ALANA had been open to any minority student. 

Starting in fall 1992, though, ALANA targeted only minority students with 2 or 3 
developmental needs (English, mathematics, or reading). Over the years, non- 
targeted minority students who had applied on their own initiative were accepted as 
well. 

Methodology 

In each fall cohort of first-time freshmen, minority students in ALANA during 
the initial fall term were compared to all other minority students. Of the 1 ,059 
ALANA students at the college from fall 1991 through fall 1994, only those students 
who were fall first-time freshmen were included in the analysis. Because prior 
college-wide cohort studies had shown developmental needs and course loads to be 
among the best predictors of achievement, students were matched on these two 
variables. Although motivation had been raised as an alternative h)q50thesis to any 
ALANA impact, it is noteworthy that students might express motivation by 
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participating in this voluntary support program. ALANA provided participant social 
security numbers. Other data came from the College's end-of-semester files and from 
the Maryland Higher Education Commission's Transfer Student System. All data 
reflected cohort populations, so inferential statistics were not used. 

Developmental Needs and Course Loads 

There was more variation among ALANA students than among other 
minority students with respect to the percent of students with 2 or 3 developmental 
needs (as assessed by the college's testing office prior to course registration). 
Depending on the fall cohort, the rate of 2 or 3 developmental needs was as high as 
83 percent and as low as 43 percent among ALANA students and as high as 43 
percent and as low as 35 percent among other minority students. In addition, the 
fraction of full-time students fluctuated more among ALANA students than among 
other minority students, with ALANA proportions ranging from 45 percent to 71 
percent (versus 37 percent to 42 percent). 



ALANA Students and Other Minority Students 
Percent of Headcount by Initial Developmental Needs and Course Loads 

Fall Cohorts 1991 through 1994 




91 


92 


93 


94 


Initial Profile 


A 


0 


A 


0 


A 


0 


A 


0 


No Need 


32 


36 


4 


36 


15 


36 


21 


34 


1 Need 


19 


21 


13 


29 


40 


27 


36 


30 


2 or 3 Needs 


49 


43 ■ 


83 


35 


45 


37 


43 


36 


Full-time 


61 


37 


63 


38 


45 


42 


71 


40 


Part-time 


39 


63 


37 


62 


55 


58 


29 


60 


HEADCOUNT 


207 


1,520 


94 


1,684 


20 


1,61 


14 


1,64 



Retention Rates 

The overall retention rate in a given term or in some subsequent term was 
higher for ALANA students than for other minority students. Among minority 
students attending college for the first time in fall 1991, 70 percent of ALANA 
students versus 53 percent of other minority students returned in fall 1992 or in some 
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later term (up through spring 1995). From this fall 1991 cohort, 46 percent of 
ALANA students and 35 percent of other minority students were enrolled in fall 1993 
or in some later term. 



ALANA Students and Other Minority Students 
Percent of Headcount in a Given Term or in Some Subsequent Term 
Fall Cohorts 1991 through 1994 
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Even when students were matched by developmental needs or by course loads, 
the retention rate in a given term or in some subsequent term was almost always 
higher for ALANA students than for other minority students. These findings held 
across cohorts. For the sake of exposition, the tables on the following page contain 
findings from the fall 1991 cohort only (the author's manuscript in the bibliography 
presents findings from the other cohorts). In fall 1992, the retention rates for ALANA 
students were 75 percent (no developmental need), 74 percent (1 developmental 
need), and 65 percent (2 or 3 developmental needs). The respective retention rates by 
developmental needs for other minority students were 50 percent, 55 percent, and 54 
percent. The fall 1992 retention rate was 77 percent for ALANA students with full- 
time course loads in fall 1991 and 59 percent for ALANA students with part-time 
course loads in fall 1991. The corresponding proportions for other minority students 
were 65 percent and 45 percent. 
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ALANA Students and Other Minority Stud 
Percent of Headcount in a Given Term or in Some Sl 
F all Cohort 1991 by Initial Developmental 
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ALANA Students and Other Minority Students 
Percent of Headcount in a Given Term or in Some Subsequent Term 
Fall Cohort 1991 by Initial Course Loads 
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Cumulative Credit Hours 



At the end of spring 1995, the average credit hours earned by ALANA 
students was least as high as for other minority students from the same cohort. In the 
fall cohorts 1991 through 1994, ALANA students earned respective averages of 23. 
credit hours, 14 credit hours, 14 credit hours, and 9 credit hours. The figures for other 
minority students in these cohorts were 15 credit hours, 14 credit hours, 12 credit 
hours, and 7 credit hours. 



Average Cumulative Credit Hours Earned through Spring 1995 
Fall Cohorts 1991 through 1994 
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Among students with developmental needs, ALANA students earned more 
credit hours (on average) than other minority students. For ALANA students with 2 
or 3 developmental needs in the fall cohorts 1991 through 1994, earned credit hours 
averaged 15 credit hours, 12 credit hours, 13 credit hours, and 7 credit hours 
(compared to 1 1 credit hours, 10 credit hours, 7 credit hours, and 4 credit hours). 
This pattern of higher earned credit hours for ALANA students than for other 
minority students held for students with only 1 developmental need as well, but the 
pattern did not always hold for students with no developmental need. 



Average Cumulative Credits Earned through Spring 1995 
Fall Cohorts 1991 through 1994 by Initial Developmental Needs 
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Full-time and part-time ALANA students did not consistently achieve more 
than their respective counterparts. Full-time and part-time ALANA students in the 
fall 1991 cohort earned an average of 5 credit hours more than their comparison 
groups by spring 1995. Yet in the fall 1992 cohort, full-time ALANA students earned 
18 credit hours, while other minority full-time students earned 22 credit hours. The 
respective numbers among part-time students from this cohort were 7 credit hours 
versus 10 credit hours. 



Average Credits Earned through Spring 1 995 
Fall Cohorts 1991 through 1994 by Initial Course Loads 
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Average GPAs 

In 3 of 4 fall cohorts, ALANA students had higher average GPAs than other 
minority students. For both sets of students, all average GPAs fell below 2.00 (the 
level required at the college for academic good standing). The highest average (a 1.88 
GPA) was earned by ALANA students from the fall 1994 cohort. 



Average GPA through Spring 1995 
Fall Cohorts 1991 through 1994 
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ALANA students with 2 or 3 developmental needs had higher average GPAs 
than other minority students with 2 or 3 such needs. For the fall cohorts 1991 
through 1994, the average ALANA GPAs were 1.57, 1.32, 1.73^ and 2.39 (compared 
to 1 .30, 1 .30, 1 . 1 8, and 1 .06 for the average non-ALANA GPAs). At other levels of 
developmental need, this pattern in favor of ALANA did not always hold across 
cohorts. 



Average GPA through Spring 1995 
Fall Cohorts 1991 through 1994 by Initial Developmental Needs 
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ALANA was not necessarily associated with the higher GPAs among fill 1 and 
part-time students. Although full-time and part-time ALANA students from the fall 
1991 cohort earned typically higher grades than their non-ALANA counterparts, this 
finding did appear with the other cohorts. Full-time and part-time students from the 
fall 1992 cohort posted generally lower marks if they were ALANA participants than 
if they were not ALANA participants (with fiill-time averages of a 1 .48 GPA versus a 
1.66 GPA and with part-time averages of a 1.35 GPA versus a 1.73 GPA). 



Average GPA through Spring 1 995 
Fall Cohorts 1991 through 1994 by Initial Course Loads 
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Prince George's Community College Outcomes Typology 

Four years after entry to Prince George's Community College, were there 
differences between ALAN A students and other minority students with regards to 
award, transfer, and academic progress Outcomes? Based on the community college 
outcomes typology that has been developed at Prince George's Community College, 
the answer is "yes." This typology has been promoted for use with cohorts having of 
minimum of four years of data and for use with regular motive students. Given 
these considerations, the table on the following page contains findings from the fall 
1991 cohort only. In addition, the table includes just those students from the cohort 
who had indicated award goals on the college application or who had attended at 
some point beyond the second term. 

Desirable outcome measures were higher for ALAN A students than for other 
minority students. Eighteen percent of ALANA students earned an award or 
transferred, compared to 10 percent of other minority students. At 13 percent, the 
transfer-only rate for the ALANA group was more than twice the transfer-only rate for 
the comparison group (at 6 percent). Among students with no awards or transfers, 
ALANA students were more likely than their counterparts to be sophomores in good 
standing (to have earned 30 or more credit hours with a 2.00 GPA or higher): The 
respective rates were 16 percent versus 1 1 percent. Similar proportions of both 
groups enrolled in spring 1995 (8 percent and 10 percent), albeit without sophomore 
good standing. Students without sophomore good standing who were not enrolled at 
the college in spring 1995 accounted for 58 percent of ALANA students and for 70 
percent of other minority students. From the perspective of the outcomes typology, 
only this last outcome measure constitutes non-achieving students. 

Even when matching students on developmental needs and on course loads, 
the desirable outcome measures remained higher for ALANA students than for other 
minority students. A brief description of these outcome measures follows (the 
author's manuscript in the bibliography presents the tabular data): Among individuals 
with no developmental need, 31 percent of ALANA students and 16 percent of other 
minority students received an award and/or transferred. Among individuals with 1 
developmental need, the corresponding proportions were 24 percent versus 9 percent, 
while these proportions were 7 percent versus about 6 percent among individuals with 

2 or 3 developmental needs. Across all three levels of developmental need, a smaller 
percent of ALANA students than of other minority students fell into the college's 
definition of non-achievers (48 percent compared to 61 percent of individuals with 
no developmental needs, 5 1 percent compared to 68 percent of individuals with 1 
developmental need, and 68 percent compared to 77 percent of individuals with 2 or 

3 developmental needs). Twenty-three percent of the full-time students and 10 
percent of the part-time students in ALANA received an award and/or transferred. 
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with respective figures of 1 8 percent and of about 5 percent for other minority 
students. While 54 percent of full-time students and 65 percent of part-time students 
in ALANA met the college's definition of non-achievers, these fractions were 63 
percent and 75 percent of other minority students. 



Outcome Measures of Regular Motive Students through Spring 1995 

Fall 1991 Cohort 
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Summary 

Over the years, Prince George's Community College has demonstrated 
commitment to its growing proportion of minority students by tracking minority 
student progress and by establishing support programs. This paper illustrated how the 
college's Office of Institutional Research and Analysis evaluated one of these 
programs through the use of a cohort analysis. 

But which program? Since minority students had increased to about 3 
quarters of all first-time freshmen and since these minority students had been 
achieving at lower levels than other first-time freshmen, the college decided upon the 
only support program- for minority students exclusively: ALANA, a program for 
African, Latin, Asian, and Native American students. Originally, ALANA had 
targeted all minority students. Starting in fall 1992, though, ALANA targeted only 
minority students with 2 or 3 developmental needs (English, mathematics, or 
reading). 

In each fall cohort of first-time freshmen, minority students in ALANA during 
the initial fall term were compared to all other minority students. Because prior 
college-wide cohort studies had shown developmental needs and course loads to be 
among the best predictors of achievement, students were matched on these two 
variables. Although motivation had been raised as an alternative hypothesis to any 
ALANA impact, it is noteworthy that students might express motivation by 
participating in this voluntary support program. In comparison to other minority 
students, spring 1995 outcomes indicated that minority students who had entered 
ALANA in their first semester: 

* Had higher retention rates 
Earned at least as many credit hours 

* Did not achieve consistently higher grade point averages 
' * Were more likely to earn awards or to transfer 

Three of these four overall findings held even when matching students on initial 
developmental needs and on initial course loads: The overall finding for earned credit 
hours held only when matching students on initial developmental needs. 
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The Institutional Researcher as Program Evaluator: 

Influencing the Development of Innovative Programs 

Anne Marie Delaney 
Director of Program Research 
Boston College 

Introduction 

Purpose . The purpose of this paper is to present a model evaluation case study 
that illustrates how program evaluation can contribute effectively to the development 
of academic and professional degree programs; how qualitative and quantitative 
techniques can be employed to produce valid and reliable measures for evaluation 
studies; and how the role of the institutional researcher can be enhanced by assuming 
the role of program evaluator. Based on an evaluation of a uniyersity-based, 
interprofessional program, the paper presents design principles and methodological 
approaches relevant to the evaluation of a broad range of academic and professional 
degree programs. 

Program Description . The program on which this evaluation study is based, the 
FIPSE InServ Project, was designed to better prepare teachers and other professionals 
to provide integrated services for at-risk children. Faculty and students from 
education, law, psychology, social work, nursing, management and arts and sciences 
collaborated to achieve the following objectives: to prepare future teachers to identify 
and address the complex educational, social, psychological, and health issues 
confronting today's children; to build interprofessional instructional teams in teacher 
preparation coursework qpd field supervision; and to establish interprofessional teams 
of students who will engage in field work/practicum experiences in schools. 

Review of the Literature . The design of this evaluation incorporated principles 
and themes from models of program evaluation in general and writings on evaluation 
of integrated services in particular. Reflecting the influence of different models of 
program evaluation, the evaluation design included a focus on program objectives 
recommended by Tyler (1949); consideration of the program's context, input, process 
and product proposed by Stufflebeam (1983); attention to the stakeholder's 
perspective advocated by Stake (1990); and the inclusion of both formative and 
summative approaches at different stages of the evaluation (Scriven, 1967). 

This evaluation was also designed to reflect the characteristics of the integrated 
services project being evaluated. Knapp (1995) recommends that evaluation of 
integrated services projects should reflect the unique and complex nature of these 
collaborative endeavors. Integrated services projects involve collaboration among 
numerous and diverse subjects. Evaluators must study the connections among the 
subjects which are often difficult to measure. Goetz (1993) proposes that the design 



of the evaluation should reflect that integrated services is an ongoing process which 
requires continuous monitoring. Gomby and Larson (1992) concur, stating that 
integrated services is a comprehensive model and study of it should be long-term and 
concentrated. 

Focusing on the complex nature of interprofessional collaboration, Levin and 
Greene (1994) assert that the nature of the collaborative endeavor implies a context of 
change, even chaos, resulting from changes in relationships, populations served, layers 
of organizations and management as well'as changing mandates for professionals. To 
capture the complex, evolving nature of integrated services projects, evaluations should 
include diverse methodologies and incorporate both process and outcome components. 

Outcome evaluations are recommended to be implemented later in the progression 
of the program, allowing the initial phase of the evaluation of a program to focus solely 
on process evaluation (Burt & Resnick, 1992). Knapp (1995) contends that attributing' 
outcomes to the project requires an understanding of the process that produced the ; 
outcomes. Consistent with these recommendations, the evaluation of this program was 
designed to reflect and support the project's developmental phases moving from 
primarily a formative, qualitative approach in the first year to a more summative, 
quantitative approach in the third year. 

Evaluation Methodology 

Development of the Evaluation Plaii . The evaluation methodology for this study 
was first outlined in the project proposal. This initial evaluation plan stated that 
evaluation activities would include monitoring progress, providing ongoing feedback 
for improvement, and assessing the impact of the project on all participants - student 
teachers, university faculty, classroom teachers, human service professionals and 
students in the classroom. Further, the evaluation would also be designed to examine 
the broader impact of the project by assessing the potential value of the project as a 
model for the reform of teacher education to represent a multi-professional approach 
to education. Once the project was funded, a more detailed evaluation plan was 
developed. 

At the outset of the project, the project evaluator met with the project 
administrators and staff to request their cooperation in the evaluation; to delineate the 
information that would be required for the implementation analysis ^d to offer 
recommendations to facilitate the conduct of the evaluation. The four primary 
recommendations offered were: to form a task-oriented evaluation committee; to 
establish a system for documenting the project activities; to rriaintain a record of all 
curriculum development work; and to schedule periodic interviews with team 
members to elicit feedback on progress and problems in implementation. The 
unifying purpose of these recommendations was to create structures and systems that 
would support the progress of the project and the success of the evaluation. Of the 
four recommendations proposed, the second and third - regarding the need for 
documentation - were formally adopted by the project staff. The evaluation assistant 
conducted the proposed periodic interviews with project participants, and the intended 
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work for the evaluation committee was handled on an informal basis between the 
project and evaluation staff. 

First Year Evaluation . Reflecting the piroject's orientation, the first year 
evaluation focused on the project’s activities, strategies and documents designed to 
establish new intellectual and institutional foundations to support interprofessional 
collaboration. The nature of this phase of the project and of the evaluation was 
primarily process oriented. The three major components of the first year program 
evaluation included: an Implementation Analysis of the Project's Activities and 
Accomplishments; a Process Evaluation on Interprofessional Collaboration; and a 
Product Evaluation on the Development of an Integrated Teacher Education 
Curriculum. 

The implementation analysis phase of the evaluation was conducted through 
interviews with various project participants; attendance at interprofessional events; 
and the collection and review of program materials, documentation, and meeting 
agenda. The following questions guided this phase of the evaluation: How did the 
project team translate the project goals into action? What activities and events did the 
project sponsor during the first year? and What was the impact of the project 
sponsored events? 

The process evaluation was of paramount importance in this evaluation as it 
related to a fundamental requirement of the project; i.e., the ability of individuals 
from different professions and segments of the educational community to collaborate 
efficiently to integrate their knowledge and skills to achieve improved services for . 
children. The following types of questions guided the process evaluation: What 
efforts were expended to create the interprofessional teams? What challenges were 
encountered in forming the teams? What strategies were employed to cope with the 
challenges? and What were the common and unique contributions offered by the 
members of the university and the school site teams? This phase of the evaluation 
was completed through observation, interviews, and reflection on the various 
collaborative experiences which occurred during the first year. 

The product evaluation involved both an assessment of progress achieved in 
creating a draft curriculum development document and a review of the modified or 
newly created curriculum in terms of the degree to which they reflected the ongoing 
dialogue between the university and school interprofessional teams; the priorities and 
concerns articulated by these teams; the professional standards of teacher educafron; 
and the feasibility concerns of the faculty who would be involved in implementing the 
curricula. 

Second Year Evaluation . The second year evaluation was intended to be both 
formative and summative. In addition to implementation, process and product 
evaluation components, the evaluation was expanded to include an outcome 
assessment of the perceived effects of project activities and events on current 
students. The specific outcomes addressed during the second year evaluation were . 
the effects of this integrated services project on the perceptions, values and 
knowledge of current education students enrolled in the revised courses. Survey 
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questions were administered to current teacher education students to elicit their , 
assessment of how well their teacher education program prepared them to achieve the 
following objectives related to the project goals: 

• to design and use evaluative procedures appropriate for students in different age, 
gender, ethnic and cultural groups; 

• to use relevant support systems within and outside the school in order to 
optimize opportunities for teaching and learriing; 

• to show respect for the unique developmental and cultural needs of special needs 
children, linguistic and other minorities; 

• to deal effectively with complexity in the classroom resulting from multiple 
teacher roles and the academic, social and moral-ethical dimensions of the 
classroom; and 

• to cope with diversity in terms of teaching students with different ability levels 
in the same class, from different socioeconomic backgrounds, from diverse 
racial/ethnic, cultural backgrounds, as well as students attending an inner-city 
school system. 

The same survey questions had previously been administered to alumni/ae who 
graduated from the teacher education program prior to the inception of the FIPSE 
InServ Project. Using Chi-Square and t-test analyses, the responses of current 
students were compared with those of the alumni/ae. 

During the second year of implementation, faculty members developed scenarios 
to be used in assessing the project's impact on current students. These scenarios are 
case studies illustrating the social-psychological problems that impact children's 
learning. Respondents are asked to identify the issues or problems reflected in the 
case; to propose strategies for addressing these issues; and to describe how they 
would collaborate with other professionals in serving these children. During the 1996 
spring semester, the original scenarios and questions were administered to students to 
assess the nature of students' responses and to identify the strengths and weaknesses 
of the scenarios and evaluation criteria. Analysis of results from this administration 
provided a basis for revising the scenarios to increase their effectiveness as evaluation 
instruments. 

The revised scenarios are being administered to students as a part of a pilot study 
during the first semester of the third year of the project. Based on results from this 
pilot study, the evaluator will conduct a reliability study and will recommend further 
revisions to ensure that these instruments meet acceptable reliability standards for use 
in the evaluation, specifically in assessing the impact of the FIPSE InServ Project on 
students' interprofessional knowledge and training. The development of valid and 
reliable assessment measures is crucial for the summative evaluation planned for the 
second semester of the third year of the FIPSE InSery Project. 
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First Year Evaluation Results 



Implementation Analysis 

The first year evaluation focused primarily on the project's implementation. This 
involved observing, recording and interpreting major events which included: an 
interprofessional academic retreat; an interprofessional campus lecture series; public 
lectures by distinguished speakers and urban seminars. A graduate student, serving as 
an evaluation assistant, assumed primary responsibility for this phase of the 
evaluation. When possible, she attended the event, or watched a videotape, 
summarized and interpreted the issues presented or discussed and noted the number of 
people participating in the event. Evaluation findings indicated that the 
interprofessional academic retreat and lecture series did provide an opportunity for 
various constituents to explore important themes in interprofessional collaboration 
and it did offer an opportunity for students, professors, and other faculty to become 
more aware of the needs of children. The evaluation report contained brief 
summaries of the project events as well as commentaries on how these events 
achieved the goals of the project. 



Process Evaluation 



As noted previously, a major goal of this project was to promote interprofessional 
collaboration. To achieve this goal, project administrators created interprofessional 
teams at the university and school sites. The process evaluation phase of this project 
included an assessment of the teams' progress; the identification of challenges 
encountered; and an analysis of the issues that emerged from the teams' collaborative 
efforts. Findings from this phase of the evaluation yielded insights relevant to the 
following aspects of interprofessional collaboration: intellectual issues, related to the 
development of an integrated curriculum; professional concerns, associated with the 
ability of individuals from different professions to cooperate effectively; and practical 
considerations, concerning the need to establish a convenient meeting site and to 
develop ongoing systems for efficient and timely communication. 

Product Evaluation 



Project participants identified four courses for revision: Child Development; 
Family, School and Society; Working with Special Needs Students; and Classroom 
Assessment. These courses were selected because they were requited for 
undergraduate education students and they encompass fundamental knowledge, 
processes, and attitudes critical to future teachers. New teaching materials were 
identified, with a special emphasis on carefully constructed case studies that have 
implications for multiple professions. 

In addition, project staff developed a new course, entitled "The Impact of 
Psychosocial Issues on Learning". This interdisciplinary course related to the goals of 
the FIPSE InServ Project in several ways. First, the course was designed to educate 
students from different professions on the importance of interprofessional 
collaboration. Second, the course focused on a variety of needs that children face, so 
participants in the class would become aware of children's educational, social. 
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psychological, and health needs, and they also would become aware of the importance 
of interprofessional collaboration for addressing these needs. 

Recommendations 

Based on findings from the previously described implementation,. process and 
product evaluations, the first year evaluation report presented the following 
recommendations designed primarily to promote improvement and enhance the , 
progress and impact of the project during the second and third year of 
implementation. The recommendations are organized according to three different 
areas of the project: interdisciplinary courses, interprofessional lecture series, and 
collaboration between the imiversity and the schools. The rationale following each 
recommendation is based primarily on evaluation findings indicating need for 
improvement. These recommendations illustrate how negative findings from the 
evaluation were transformed into positive, constructive recommendations for 
improvement. 

Interdisciplinary Courses. 

1. Ensure that interdisciplinary courses reflect a balanced perspective . 
Interprofessional academic preparation is essential to the preparation of future 
professionals who plan to engage in interprofessional collaboration. Such courses 
need to reflect multiple disciplinary perspectives and students should be required to 
take these courses. To strengthen the collaborative experience, such courses might 
include a course project to be completed by students from different professional 
disciplines. This course related experience would potentially expand the 
interprofessional collaborative training opportimities for students and prepare them 
for their work at the school sites. Also students might develop case studies based on 
their interprofessional collaboration during their practicum. 

2. Encourage faculty to integrate the lecture series with their courses . More 
students may attend the lectures if they are required as part of regular course 
activities. In this way students will become more aware of the need for and value of 
interprofessional collaboration. Results from recent alumni/ae research conducted at 
this institution support efforts to share the goals and ideas of this project with all 
students who are preparing to work with children, particularly in a school setting. 
Based on their ovm professional practice, our alumni/ae advocate intensive pre- 
professional preparation to enable newly practicing professionals to cope with the 
critical social-psychological challenges they will encoimter in serving children at risk. 
This preparation can only be enhanced by attending the interprofessional lectures. 

3. Offer an interdisciplinary seminar course on the case study method . Given the 
importance of case studies in interprofessional training, service and evaluation, it may 
be beneficial to offer a series of seminars or workshops on the case study method 
early in the second year of the project. Such a series might address both the intrinsic 
and instrumental value of the case study method for social work, education and 
program evaluation. Faculty, graduate students and other project participants would 
potentially benefit from such a seminar. 
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Interprofessional Lecture Series. 

4. Increase the publicity about each lecture . If it has not already been done, 
explore various channels of communication in the university - including the faculty 
biweekly paper and the student newspaper. Publicizing the project would result in 
more professionals attending the lectures, and therefore, becoming more aware of the 
needs of children. 

5. Consider holding some of the lectures, retreats, and workshops at the school sites . 

6. Solicit evaluative data for ongoing program planning . A brief evaluation form 
might be distributed at each lecture. Participants might be asked: What was your 
goal in attending? To what extent was your goal achieved? and How has the 
presentation influenced your perspective on interprofessional collaboration? 

Collaboration between the University and Schools . 

7. During the second year, focus intensive efforts on parental and community 
involvement in the project . Since so much progress has been made already in 
achieving the goals of the University Collaboration Program, more time and resources 
might be expended in the second year on the Public School Services Program, 
particularly in attempting to increase parental and community involvement in the 
project. 

8. Increase communication and establish strong relationships w ith representatives 
of community agencies . The FIPSE InServ Project team should intensify efforts to 
assure professionals in the community service agencies of the project's intent to 
support, not eliminate, their roles at the school site level. Proposed strategies include 
communicating more often with the agency directors; inviting staff to conferences and 
meetings; providing compensation for their involvement in the project; and 
strengthening efforts to establish a spirit of mutual trust and support. 

Second Year Evaluation Results 

As noted previously, the second year evaluation continued to serve a formative 
role, documenting the project's implementation and offering recommendations for 
program improvement. In addition, the evaluation was expanded to include an 
assessment of the project's impact on students enrolled in the new curricula. This 
assessment was based on comparative analyses of student survey results documenting 
the perceived effect of the FIPSE InServ Project on the preparation of current and 
former teacher education students as well as on analyses focusing only on current 
students' evaluation of how well the FIPSE InServ Project enabled them to achieve 
specific goals. Selected results from the comparative segment of the evaluation 
follow. 

During the 1996 spring semester, a survey was administered to students enrolled in 
the four FIPSE related courses. FIPSE student responses were compared with those 
of 'non-FIPSE students', i.e., alumni/ae who were enrolled in teacher education 



programs from 1987 through 1993, prior to the inception of the FIPSE InServ Project. 
The common questions, administered to both groups of students, related to how well . 
the teacher preparation program prepared students to fulfill multiple teacher roles, to 
utilize relevant support systems, and to respond effectively to the unique and diverse 
needs of the students they teach. 

As shown in Table 1, compared with the non-FIPSE students, the FIPSE students 
evaluated their teacher preparation program more positively in terms of preparing 
them to assume multiple teacher roles in order to cope with complexity in the 
classroom and deal with the academic, social and moral-ethical dimensions of the 
classroom. Some 36 percent of the FIPSE students, compared with only 22 percent of 
the non-FIPSE students, rated the program 'Very Well' on this objective. As indicated 
by the Chi-Square statistic, the difference between the FIPSE and non-FIPSE student 
ratings is statistically significant. 



Table 1. FIPSE and Non-FIPSE Students' Evaluation of 
their Preparation to Serve Multiple Teacher Roles 



Student Group 


Poorly- 
Very Poorly 


Fair 


Well 


Very Well 


Total 


FIPSE 


1.5% 


13.3% 


48.9% 


36.3% 


100.0% (N= 13 5) 


Non-FIPSE 


9.1 


31.8 


36.9 


22.2 


100.0% (N=320) 



^ = 30.94 prOOl 

FIPSE students also offered significantly more positive ratings in terms of the 
preparation offered for using relevant support systems, within and outside the school, 
in order to optimize opportunities for teaching and learning. Seventy-two percent of 
the FIPSE students, compared with 59 percent of the non-FIPSE students rated the 
program 'Well' or 'Very Well' in relation to this objective. The statistically significant 
results are presented in Table 2. 



Table 2. FIPSE and Non-FIPSE Students' Evaluation of 
their Preparation to Use Relevant Support System 



Student Group 


Poorly- 
Very Poorly 


Fair 


Well 


Very Well 


Total 


FIPSE 


3.1% 


24.8% 


45.7% 


26.4% 


100.0% (N=129) 


Non-FIPSE 


9.2 


31.9 


39.6 


19.3 


100.0% (N=326) 



9.16 pros 

Another major goal of the FIPSE InServ Project was to prepare teachers to be 
sensitive and responsive to the unique needs of individual children and special student 
groups. Such sensitivity requires professional insight and the capacity to design 
instructional strategies and assessment procedures suitable for children in different 
groups. Evaluation results revealed that FIPSE students rated the program 
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significantly higher than did non-FIPSE students in terms of the preparation received 
to respect the unique needs of special needs and minority children (X2=i6.55 p ^ 

.001). Some 54 percent of the FIPSE students, compared with 40 percent of the non- 
FIPSE students, rated the program very positively on this objective. Similarly, a 
significantly higher percent of the FIPSE students also rated the program positively 
with respect to the preparation received to design and use evaluative procedures 
appropriate for children in different groups (X2=17.28 p^ .001). Over 80 percent of 
the FIPSE students, compared with only 65 percent of the non-FIPSE students, rated 
the program 'Well' or 'Very Well' on this objective. 

Maximizing students' potential for academic success, a primary goal of the FIPSE 
InServ Project, requires that teachers are prepared to cope with various aspects of 
diversity in the children they teach. Chi-Square results revealed the greatest perceived 
difference in the preparation for teaching students from diverse racial/ethnic and 
cultural backgrounds (X2=59.23 p" .001). Approximately 80 percent of the FIPSE 
students, compared with only 47 percent of the non-FIPSE students, rated their 
teacher preparation program 'Well' or 'Very Well' on this objective. Results also 
showed that FIPSE students consistently rated their teacher preparation program 
significantly higher, than did non-FIPSE students, in terms of its effectiveness in 
preparing them to teach students with different ability levels (X2=l 1 . 1 3 p^ .05) and 
from different socioeconomic backgrounds (X^=43.70 p^ .001). 

In addition to the individual item analysis, two scales were created to produce 
more reliable and succinct measures of perceived preparation for assuming multiple 
teacher roles and teaching diverse students. The first scale. Preparation for Multiple 
Teacher Roles is based on students' ratings of two items: preparation for dealing 
effectively with complexity from multiple teacher roles and preparation for using 
relevant support systems to optimize learning. The second scale. Preparation for 
Teaching Diverse Students included six items referring to the preparation to respond 
to children's unique and diverse characteristics in ability and socioeconomic, 
racial/ethnic and cultural background. The statistical properties of these scales are 
presented in Table 3. As shown, the reliability for the first, two-item scale is 
moderate, .68, and the reliability for the second, six-item scale is high, .88. 



Table 3. Statistical Properties of the FIPSE InServ Project Evaluation Scales 



Scales 


Mean 


S.D. 


Reliability 


No. of 
Items 


Range of 
Responses 


Preparation for Multiple Teacher Roles 


3.81 


.06 


.68 


2 


Low - High 
1-5 


Preparation for Teaching Diverse Students 


3.75 


.30 


.88 


6 


1 -5 



Table 4 presents group mean ratings and the results of the t-test corriparing FIPSE 
and non-FIPSE students' ratings on the scale. Preparation for Multiple Teacher Rol es. 
As shown, the FIPSE mean rating is significantly higher than that of the non-FIPSE 
student rating. 



2 



B 



21 



Table 4. Comparison of Student Group Means on their 
Preparation for Multiple Teacher Roles 





FIPSE Students 


Non-FIPSE Students 


t Value 


X 


4.07 


3.70 


. . 5.34 


S.D. 


(.58) 


(.83) 


(p^OOl) 


N 


127 


308 





Table 5 presents FIPSE and non-FIPSE students' mean ratings on the scale. 
Preparation for Teaching Diverse Students . As indicated by the t value, the FIPSE 
mean rating Of 4.13 is significantly higher than the non-FIPSE rating of 3.62. 



Table 5. Comparison of Student Group Means on their 
Preparation for Teaching Diverse Students 





FIPSE Students 


Non-FIPSE Students 


t Value 


X 


4.13 


3.62 


7.35 


S.D. 


(.60) 


(.79) 


(p^OOl) 


N 


128 


3II 





Qualitative data were also used to document the perceived effect of the FIPSE 
InServ Project. These data generally supported the quantitative data documenting the 
perception that FIPSE related courses were preparing students to identify and address 
the complex educational, social, psychological, and health issues that confront today's 
children and impede their learning. Some students, however, qualified their positive 
evaluation by noting that they were in the early stages of their teacher preparation 
program and would need more time, education and experience to be prepared. 
Representative comments included in the evaluation report follow. 

"I have learned that there are many obstacles that can impede a child's 
learning. I am ready and willing to deal with them in the best way I 
know how." 

"I feel that I am becoming better prepared to address and deal with the 
many issues that confront today's children. However, until you are 
confronted with the problems face to face, your understanding and 
preparation is somewhat limited." 

"I feel very confident that I know about the various issues that face 
children today. By discussing them and seeing different sides to these 
issues in my classes, I have developed ways that I would address these 
issues as a teacher. I would not say yet that I am fully qualified to 
handle all of student issues specifically because I still need more 
experience." 
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Several students also commented positively on their preparation to collaborate with 
other professionals to meet children's needs. Illustrative comments follow. 

"I believe that after our in-class, group project, I will be very well 
prepared. I will be exposed to the roles of many other professionals and 
how they affect a child's life. I will better understand how to talk with 
them and what to discuss with them." 

"We are doing a project in which collaboration among lawyers, social 
workers, doctors, and teachers is the goal. This is effective in preparing 
us for later collaboration with other professionals." 

"I feel very prepared in being able to collaborate with other professionals 
in order to come up with solutions to problems facing children." 

Recommendations 

Recommendations presented in the second year evaluation report identified two 
areas for priority attention: interprofessional curriculum development and parental 
involvement. The proposals regarding curriculum development reflected feedback 
received from the faculty during the course of the evaluation. 

Interprofessional Curriculum Development 

1. The FIPSE InServ Project should increase faculty involvement in the 
development of interprofessional curricula . As noted in the project proposal, course 
instructors should meet regularly with the Project Co-Directors in order to share 
teaching/1 earning experiences regarding the FIPSE related courses. Now that the 
FIPSE InServ Project has created the context for interprofessional education and 
training, it may be beneficial to focus more intensively on specific aspects of 
curriculum development. Initiatives, proposed by some faculty, include creating a 
systematic plan to transform the curricula; supporting this plan with relevant 
resources, such as a list of guest speakers for classes and video footage of full-service 
schools; and providing ongoing leadership and guidance in the development of case 
studies and other curricula materials relevant to interprofessional training and 
integrated services. The newly developed interprofessional curricula for pre-service 
teachers should emphasize the application of theory to practice and incorporate real 
life' situations in the classroom. 

Parental Involvement 

2. The FIPSE InServ Project should continue to reach out and attempt to increase 
parents' involvement in their children's education . A first step to achieving this goal 
may be to identify existing barriers to parental involvement in the schools. Strategies 
could then be developed to overcome these barriers and facilitate parents' 
participation in their children's education and in school events. Through increased 
communication and responsiveness to parents' concerns, the FIPSE InServ Project 
might realize the ideal goal of collaborating with parents in promoting their children's 
development. 
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Discussion 



This paper addresses a number of issues that one might encounter in the 
evaluation of any academic or professional degree program. The first issue is the 
importance of utilizing theory in the construction of the evaluation design; several 
theoretical models of program evaluation were incorporated in the design for this 
study. A second fundamental and generalizable issue is the essential role of 
implementation analysis in assessing the program's impact. A valid inference about 
outcomes requires sufficient documentation of the program's implementation. 

This study demonstrates how an evaluation can reflect a project's developmental 
phases by progressing from a formative to a summative approach. The 
implementation analysis illustrates how one can transform daily activities and events 
into a chronicle of accomplishments. The process evaluation manifests how a general 
theoretical or conceptual approach can provide an organizing structure and illuminate 
lessons to be learned from an evaluation. The second and third year evaluations 
identify two methodologies - surveys and case studies - potentially relevant to the 
outcome phase of an evaluation. Finally, the recommendations illustrate how 
negative findings can be transformed into constructive proposals for improvement. 
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TUITION AND FEES AT PENN STATE FROM 1 859 TO 1996 



Michael J. Dooris 
Director of Academic Assessment 
The Pennsylvania State University 
417 Old Main 
University Park, PA 16802 



O. Richard Bundy, III 
Assistant Director of Annual Giving 
The Pennsylvania State University 
101 Rider House 
University Park, PA 16802 



The rising cost of attending college is a sensitive topic for higher education. 
This paper introduces the broad policy issues in a national context, then examines the 
experiences of a single institution — Penn State University. 



Articles about scary college costs are a staple of popular magazines and 
newspapers. But those stories can be misleading. They often feature prices at the 
finest and most expensive institutions, which of course do have a disproportionate 
share of the nation’s best students, scholars, and influence. But no single sector is 
representative of U.S; higher education. Elite private institutions only enroll about 
three percent of America’s undergraduates. Seventy-five percent of the nation s 
enrollments are in public higher education. The universities of the Big Ten alone 
grant 20 percent of the nation’s doctorates, 15 percent of its masters degrees, and 8 
percent of its baccalaureates. Also, the popular press usually discusses sticker prices, 
not real costs to students. Typically, prominent private institutions have an effective 
discount rate (based on grants from institutional funds) of about one-third of 
published tuition and fees. Substantial discounting occurs at other types of 
institutions, as well. For all four-year institutions, about one-half to three-fourths of 
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students typically receive aid (the numbers vary by type of institution). Total 
institutional aid and federal aid have both increased in recent years. And public 
colleges’ charges for tuition, room, and board have been essentially flat, in constant 
dollars, over the past thirty years (General Accounting Office, 1995; Hubbell, 1995). 

It is also clear that the U.S. system works. The number of Americans enrolled 
in college is at an all-time peak. The percentage of citizens graduating from college is 
the highest in the world. Compared to high school graduates, college graduates earn 
about two-thirds more, experience lower unemployment rates, and enjoy a lifetime 
annual return (taking into account indirect costs, foregone income, and so on) of about 
13 percent on their investment in a baccalaureate degree (Leslie & Brinkman, 1988). 
The alarms about higher education are exaggerated. 

On the other hand, a 30-year national survey of entering freshmen showed 
those entering in Fall 1995 were more worried about college affordability than any 
entering class since 1 966 (Sax et al., 1 995). Are those concerns justified? In some 
respects, yes. However, questions about this topic are difficult to answer categorically. 

Important indicators in this area can support different and even contradictory 
conclusions. For example, the inflation-adjusted cost of attending a public college 
has held steady since the mid-1960s. However, at private institutions, tuition and fees 
have outpaced inflation for just about any period in the past five to thirty years. For 
example, the sticker price at private institutions rose by 4.5 percent per year in 
inflation-adjusted dollars from 1980 to 1990 (Clotfelter, 1996). And even at public 
institutions, tuition and fees outpaced inflation for much of the 1 980s and early 
1990s — a period in which tuition helped compensate for stagnant or declining state 
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appropriations. The trend of declining state support may have struck bottom in 1992 
and 1993 (Hines & Higham, 1996). 

The changes are complex and inter-related. The student share of costs 
doubled from 17 percent to 34 percent, from 1950 to 1990, even though the total 
family share essentially held steady. This reflects the growing numbers of adult, 
independent students, and the shift away from federal aid in the form of grants toward 
loans (Hauptman, 1993). While federal student aid is substantial and continues to 
grow, the mix of grants and loans has changed from about 75 percent in grants to 
about two-thirds in loans since the 1970s. Researchers believe that rising tuition 
rates, changing patterns of aid, and increased net costs to students have diminished 
college access, graduation rates, and college choice. These concern pertain not 
exclusively, but especially, to low-income students (General Accounting Office, 

1995; Mumper, 1996). 

In sum, the big picture of causes and effects is difficult to interpret. The family 
share of cost may not have changed, but the share borne by students has doubled. 
Prices have increased sharply — except for some sectors and some time periods. 
Participation is high and increasing, but access and participation of low-income and 
under-represented students and college choice for many students have suffered. In 
view of the complexity of aggregate data and trends, it is reasonable to look elsewhere 
for insight, as well. 

The Institutional Context 

Economist Charles Clotfelter’s highly regarded 1996 book. Buying the Best , 
provided detailed analyses of four prestigious private institutions (Chicago, Duke, 
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Harvard, and Carleton). It is an excellent demonstration that, given the state of the art 
of the economics of higher education, case studies are necessary, appropriate, and 
informative. Broad trend data and public policy discussions about college cost, 
access, and affordability are fine. But understanding also requires detailed, 
disaggregated, institutional data, and interpretation in the context of a specific 
university’s history and circumstances. 

Tuition is one factor in institutional management, one part of the planning and 
budget balancing act. Included in that balancing act are enrollments, financial aid, 
instructional workload, classroom capacity, faculty and staff salaries, plant 
maintenance, purchases of books, instrumentation, and computers. Especially in 
relation to matters such as these, tuition is increasingly viewed pragmatically. There 
is an emphasis on maximizing revenue and/or maintaining or promoting market share, 
through strategies such as pre-payment plans, user charges, cost pricing, and 
differential pricing. Beyond these relatively mechanical, essentially budgetary issues 
are more amorphous, but nonetheless critical, concerns. These include vulnerability 
to charges of greed, bloat, and unresponsiveness; the capacity to support the creation 
and extension of knowledge through research and graduate education; the size, shape, 
and character of the curriculum; and the rise or fall of institutional prestige. State- 
supported institutions must also pay attention to their ability to carry out the mission 
of providing public access to high quality education. 

Unexamined Questions about Tuition History 
Any well-managed college or university has considerable self-knowledge. 
Penn State is no exception. The institution has considerable data on emollments 
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(72,000 and rising); tuition and fees ($5,258 at the main campus, also rising); the 
percentage of students receiving financial aid (78 percent); the average loan debt of 
baccalaureate students at graduation ($13,000); the default rate (4.7 percent); rates of 
tuition increases (average of 8.9 percent from 1967-68 through 1995-96); faculty 
salaries ($77,600 for full professors at the main campus); how these figures compare 
to those of peer institutions; and much more. 

However, few institutions have a long-term perspective of their tuition. Until 
recently, Penn State was unable to answer clearly questions such as the following. 

How have charges for and policies about tuition, room, and board changed over the 
history of the university? How do the changes compare to changes in the costs of 
other goods, services, and wages? How does the price tag from the earliest days 
compare to the price of today? Are recent charges unreasonable by historical 
standards? 

A Recurring Theme 

The 60 boys who entered Pennsylvania’s Farmers High School in February 
1859 each paid $100 to cover their educational expenses for the institution's first 
academic year. That $100 covered room and board, tuition, washing; and fuel; 

students were required to furnish and clean their own rooms. In 1 864, those fees were 
raised to $200 per year. That was Penn State's first fee increase, and in a sense it 
remains the biggest. Never again have charges been doubled at a single stroke. (This 
example also illustrates the difficulty of tracking charges over time. There is no 1996 
equivalent for the “room and board, tuition, washing, and fuel” of 1859. More will be 
said on limitations soon.) 
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Dissatisfaction with costs has been a recurring theme. In 1 897, chemistry 
students accused the college of “discrimination,” because they were required to pay 
practicum fees for laboratory courses while other students enjoyed free tuition. In 
1961, the university’s president publicly warned that tuition (then $525 per year) 
could not go much higher “without seriously affecting the ability of our young people 
to go to college.” In 1986, the president of the university responded to student 
pressure by appointing the President’s Advisory Committee on Alternatives to Tuition 
Increases (there weren’t many). The current president convened a task force in 1996 
to consider possible revisions to the tuition structure. Today’s concerns about tuition 
are not completely new. 

Data Limitations 

How does Penn State’s $100 price tag of 1 859 compare to today's price (for 
tuition, room, and board) of $9,558? 

This is not a simple question to answer. For example, from 1 859 to 1 878, 
students were either required or given the option to work (the policy varied during 
those early years) in the fields, shops, and bams to keep charges low. From 1 872 to 
1 960, the word “tuition” was officially inaccurate, since there was technically no 
tuition charged to Pennsylvania residents. During most of that period, however, 
students were billed for “incidentals” in addition to room and/or board and various 
other items, such as laboratory and equipment fees. The fees approached room and 
board charges in size, however, and were of a magnitude well beyond "incidental." In 
1960, the university, for the first time in the twentieth century, officially set a charge 
for tuition. 
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The handling of room and board at Penn State also varied considerably over 
the years. For most of Penn State's pre- 1950s history, room and board were not 
offered as a package. In the 1920s, meals were offered as an option in addition to 
dormitory rooms, but to women students only. During the 1920s, 1930s and 1940s, 
most men lodged off campus, and those who lived in dormitories made their own 
arrangements for meals at downtov/n restaurants, boarding houses, eating clubs, or 
fraternities. 

For these reasons and others (such as calendar revisions, the imposition of 
various fees, and differential tuition rates), it is impossible to develop perfectly 
comparable charges for room, board, and tuition. It is also difficult to, precisely 
correlate changes in tuition levels to broader price shifts over 136 years. An earlier 
analysis (Dooris, 1989) provided the methodology for working within the limitations 
of the available data, and also provided detailed data that do hot fit within the scope 
of this present paper. Although variations over the years make precision an elusive 
goal, reasonable comparisons are possible. 

Consumer Prices 

The movement of Penn State's fees for tuition and room and board over the 
past 1 36 years has been fairly consistent in direction with general inflationary or 
deflationary trends, although the magnitude of shifts has been greater. Figure 1 shows 
how Penn State’s tuition has related to the Consumer Price Index, especially since 
1945. While consumer prices rose by 75 percent from 1945 to 1965, Penn State’s 
tuition, room, and board costs rose by 140 percent: at about a two-to-one rate. From 
1965 to 1995, the CPI rose by about 380 percent, and Penn State’s charges rose by 
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650 percent: again, roughly two-to-one.. From 1986 through 1995, the CPI went up 
41 percent while Penn State’s charges increased 74 percent. (Although the CPI is not 
an ideal indicator, it is a consistent and usable benchmark for this analysis.) 

During the past 136 years, Penn State’s charges have increased at more than 
twice the rate of inflation — 945 percent as opposed to 392 percent. Throughout, Penn 
State’s charges have risen more quickly than the CPI during inflationary periods, and 
fallen more slowly during deflationary periods. In short, the recent relationship 
between Penn State’s increases and inflation rates in recent decades and in the post- 
World War II period is fairly consistent with the approximate two-to-one relationship 
that has existed for 136 years. 

Figure 1. Tuition and Consumer Price Index 
Percent Change of Tuition vs. CPI - Historical 
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Tuition Relative to Other Institutions 

When the Farmers High School opened its doors in 1 859, in the infancy of 
public higher education, its tuition was about one-third the rate charged by most other 
colleges in Pennsylvania. Well into the 1920s, when Penn State had established itself 
as a legitimate modem college, tuition rates were one-half to one-third those of the 
most expensive institutions. In 1928, when Penn State's tuition was $132, tuition at 
Temple (at the time, privately controlled) was $215; the University of Chicago, $270 
to $300 per year; Boston University, $300; and Yale, about $350 per year (Robertson, 
1928). Similar data for 1945 show Penn State at about one-third the cost of an Ivy 
League or other top-tier private institution (Good, 1946). By the 1995-96 academic 
year, Penn State, with in-state tuition of $5,258, compared to Bucknell at $19,730; 
Boston University at $19,700; Yale at $21,000; Chicago at $20,193; and Harvard at 
$20,444 (U.S. Department of Education, 1995). Costs at elite private institutions ran 
up especially rapidly in the 1 980s. Relative to those universities, Penn State is now a 
bit less expensive — at about one-fourth the tuition — than at any other time in history. 

Income Levels 

Income levels are more relevant to changing college costs, and the ability to 
pay those costs, than a market-basket price index. Penn State’s tuition as a percentage 
of the income of households headed by a college graduate has trended upward slowly, 
within the range of about four to eight percent. In 1939, tuition ($135) was five 
percent of earnings, at $2,600; in 1985, tuition ($2,760) was six percent of earnings, 
at $46,000; and in 1993, tuition ($4,822) was eight percent of earnings, at $56,000 
(U.S. Bureau of the Census, series; U.S. Bureau of the Census, 1975). From 1964 to 
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1995, tuition at Penn State and average U.S. salaries grew at virtually identical 
rates — 488 and 489 percent, respectively (U.S. Department of Labor Statistics, 1995). 

Faculty Salaries 

Penn State’s charges for tuition,. room, and board have usually been one-sixth 
to one-eighth professors’ average salaries. In the 1860s, professors in large 
institutions earned average salaries of about $1,000 per year, while Penn State’s 
charges were about $100 to $200. In 1929, the average faculty salary was about 
$3,100 (U.S. Bureau of the Census 1975) — approximately seven times Penn State's 
charges. For 1995-96, the average salary for professors in doctoral institutions of 
$73,610 (The Chronicle of Higher Education , 1996) was about eight times the $9,558 
price of tuition, room, and board at Penn State. 

Other Indicators 

Similar comparisons can be made with other historical cost and income 
measures. Penn State tuition has become less expensive, for example, in relation to 
farm values and farm incomes over the past century, and less expensive in relation to 
health care costs in the past 50 years. (For more detail, see Dooris, 1989.) 



Conclusions 

Two notions emerge from this analysis. First, the combination of trends for 
aid, tuition, and college attendance do not portray a system that is obviously out of 
control nationally. This appears true for the particular institution examined closely 
here, as well. The cost of a Penn State education today is not exorbitant, by the 
historical standards established during the past 136 years. In many respects, the 
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university’s cost has maintained a consistent relationship to inflation rates, and to 
average income levels, faculty salaries, and tuition at top private institutions. 

The second point, however, is this. Any complacency that might have 
accompanied such a conclusion ten or fifteen years ago is hard to justify today. 
Shifting federal aid patterns, reductions in the proportion of state-appropriated 
support for public institutions, concerns about college access and affordability, and 
increasing pressures for productivity and accountability have changed the 
environment. Penn State — no doubt like many public universities — will want to 
continue exploring aggressive and innovative approaches to matters of tuition, student 
aid, productivity, cost control, and institutional quality. 



O 



37 



41 



Bibliography 



The Chronicle of Higher Education (1996, September 2). Almanac issue, (43) 1 . 

Clotfelter, Charles T. (1996). Buying the best: Cost escalation in elite higher 
education . Princeton, NJ: Princeton University Press. 

Dooris, Michael J. (1989, February). Tuition and fees at Penn State: 1859-1989. 
University Park, PA; Office of Planning and Analysis. 

General Accounting Office (1995, March). Higher education: Restructuring student 
aid . Washington, DC: U.S. General Accounting Office. 

Good, Carter V. (Ed.). 0946 1. A guide to colleges, universities, and professional 
schools in the United States . Washington, DC: American Council on Education. 

Hauptman, Arthur M. (1993, February). Higher education issues in the early 1 990s . 
New Brunswick, NJ: Consortium for Policy Research in Education. 

Hines, Edward R., and Higham, J. Russell (1996). State appropriations for higher 
education, 1996-1997. Normal, IL: Illinois State University. 

Hubbell, Loren Loomis (1995, February). Tuition discounting is on the rise. 
NACUBO Business Officer, pp. 27-3 1 . 

Leslie, Larry L., and Brinkman, Paul T. (1988). The economic value of higher 
education . New York: MacMillan/ American Council on Education. 

Mumper, Michael (1996). What went wrong and why: Reviewing the causes of 
declining affordability. In Removing college price barriers: What government has 
done and why it hasn’t worked (pp. 215-228). Albany: State University of New 
York Press. 

Robertson, David Allan (Ed.) (1928). American universities and colleges . New York 
and London: American Council on Education, Charles Scribner's Sons. 

Sax, L.J., Astin, A.W., Kom, W.S., and Mahoney, K.M. (1995, December). The 
American freshman: National norms for Fall 1995 . Los Angeles: Higher Education 
Research Institute. 

U.S. Bureau of the Census (series). Current population reports . Washington, DC: 
Government Printing Office. 

U.S. Bureau of the Census (1975). Historical statistics. Colonial times to 1970 . 
Washington, DC: Government Printing Office 




38 



42 



U.S. Bureau of Labor Statistics (1995, December). Monthly Labor Review . “Current 
Labor Statistics; Average Hours and Earnings by Industry.” Washington, DC: 
Government Printing Office 

U.S. Department of Education (1995). Tuition and required fees and room and board 
charges at 4-vear. 2-vear. and public less-than-2-vear institutions . Washington, DC: 
Office of Educational Research and Improvement. 



O 



39 



43 



Who Does the Community College Serve? One College’s Experience 

(A Workshare) 

Eleanor Fujita 

Director, Academic Information 
City University of New York 

All colleges provide profiles of their student body. However, while such a 
profile provides a snapshot of who is attending the college, it does not provide any 
basis for interpreting how well the college may be functioning in terms of enrolling 
persons in the service area. Understanding how the institution is serving various 
groups within the population is of particular importance to community colleges which 
were founded to serve specific geographic areas, and especially, for urban community 
colleges serving diverse populations. It is also important that the resulting data be 
comprehensible to fellow administrators, planners and colleagues. 

In order to explore the question of how well the college was enrolling its 
constituent residents, the research office at an eastern urban community college 
calculated, what staff termed, “service rates” based on a variety of demographic 
characteristics. These rates or indices provided the research office a standard measure 
for comparing the college’s service to different groups within the population which 
could be easily understood by colleagues. 

The college had previously been evaluated by state officials who used a single 
index which was essentially the percentage of county residents at least eighteen years 
of age who were enrolled in a given fall term (penetration or “market penetration” 
rate). Further the state officials used this one statistic, without any context, and 
compared the community colleges with one another. The results were unproductive. 
The statistic was an abstract one which did not in any way aid officials at any of the 
colleges in understanding how their particular college was fimctioning in its 
environment. In fact, the use of the measure needlessly encouraged an attitude that 
suggested some colleges were doing a fine job and others a poor one. In fact, no one 
really knew if this was true or not. No college was given a handle to understand how 
it was doing in enrolling those who needed a college education. No consideration was 
given as to whether there were other opportunities within a county (a state or private 
college) for residents to obtain a college education, whether the adult population 
already had a college education or not, or whether the residents could afford the time 
and money it took to further their education. The abstract but simplified index 
simply did not serve a very useful purpose. 

The index would have cleared one hurdle if it had been used to take context 
into consideration, but the abstract nature of the statistic would still have presented a 
problem. The calculations often resulted in numbers less than one percent, and 
expressed to two decimal places they were quite abstract and difficult to comprehend. 
Since we were in the business of serving people, it did not make a lot of sense to say 
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we were serving the community at a rate of .06. We found that by using a base of 
10,000 rather than 100 we could express the results in terms of whole persons. And 
further, we decided that if we called the results “service rates” we would better 
describe the business we were in. 

The service rates were calculated essentially the same as a percentage (or a 
market penetration rate) but using a larger multiplier: we divided the number of 
enrolled students eighteen years of age or older by the number of residents who were 
also eighteen or older and then multiplied the results by 10,000. 

Our first goal was to learn what portion of our population we were serving and 
to be able to express it in a way that would make sense to the college community. 

Our second goal was to learn whether, as time passed, we were serving more or less 
of our population from year to year, since recruiting and retention were being 
emphasized. Our third, and perhaps most important, goal was to consider our 
context, to find how we were doing at serving different groups of residents within the 
county. Essentially, we wanted to learn who we were serving, and conversely, who 
we were not serving. To do this we calculated service rates for each municipality 
within the service area. Then, within each municipality we would calculate service 
rates for according to educational achievement level, race/ethnicity, gender, age, and 
finally on a combination of factors. And, our final goal was to present the results to 
the college community. This we did by treating the entire project of finding who the 
college was serving as a mystery to be solved. Each month, in the staff newspaper 
new evidence was presented until we got closer and closer to solving the mystery of 
just who it was the college was serving— and who it was not serving. With these goals 
in mind, what was it that we found? 



The Results 



Over Time: 

In the Fall of 1990, the college was enrolling 62 of every 10,000 residents 
aged eighteen years or older. However, by the Fall of 1994, after five years of steady 
growth, the college was serving 40 percent more residents: 87 of every 10,000. 



Five Year Enrollment and Service Rates* for All Residents 18 Years of Age and Older 


Term 


Enrollment 


Service Rates* 


Fall 1990 


2685 


62 


Fall 1991 


2732 


63 


Fall 1992 


2948 


68 


Fall 1993 


3307 


77 


Fall 1994 


3755 


87 




42 



45 



*Note: Service Rates based on population eighteen years old and over (430,695) as 
determined by 1990 U.S. Census; service rates were derived by dividing the 
enrollment by population eighteen years of age and older and then multiplying by 
10 , 000 . 

The Municipalities: 

We concentrated on the Fall 1994 enrollment for a more thorough analysis. 
As we calculated the service rates for the twelve municipalities within the service 
area, the wide range of service became evident. In one municipality we were serving 
a high of 1 13 of every 10,000 residents, and in another we serving a low of only 22 
persons of every 1 0,000. The two main sites of the college were located within or 
very near the five municipalities who were served at the higher-than-average rate. 
Classes were held in the high schools of outlying municipalities, but service rates 
were predictably lowest for those areas where few or no classes were offered. 



Service Rates’*' Fall 1994 for All Residents 18 Years of Age and Older by Municipality 


Municipality 


Population 18 Plus 


Enrollment 


Service Rates’*' 


Municipality # 1 


11,830 


26 


22 


Municipalities # 2 & 3 


12,254 


34 


28 


Municipality # 4 


27,420 


96 


35 


Municipality # 5 


27,815 


141 


51 


Municipality # 6 


49,284 


293 


59 


Municipality # 7 


38,516 


316 


82 


Municipalities # 8 & 9 


36,704 


362 


99 


Municipalities # 1 0 & 1 1 


54,169 


543 


100 


Municipality # 12 


172,703 


1944 


113 



*Note: Service Rates based on population eighteen years old and over as determined 
by 1990 U.S. Census. 

Achievement Level: 

As mentioned previously, the college was enrolling 87 of every 10,000 
residents in the county in the Fall of 1994. Since the primary mission of the college 
was to offer programs leading to degrees, it seemed appropriate to consider the 
college’s success in enrolling residents who had not yet attained a college degree. 
When the service rate was calculated for this group of residents for the Fall term of 
1994, it was found that the college was serving 1 13 of every 10,000 such residents — 
the service to this group was 30 percent higher than to the general population. Since 
almost none of the enrolled students entered with a college degree already in hand, the 
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service rates which increased each year were considerably higher than for the service 
to the general population without regard to educational achievement. 



Five Year Enrollment and Service Rates* for All Residents 18 Years of Age and Older 

Without a College Degree 


Term 


Enrollment 


Service Rates* 


Fall 1990 


2685 


81 


Fall 1991 


2732 


82 


Fall 1992 


2948 


88 


Fall 1993 


3307 


99 


Fall 1994 


3755 


113 



*Note: Service Rates based on population eighteen years old and over (333,451) 
without a college degree using figures from the 1990 U.S. Census. 

We then looked at the service to those without college degrees in each 
municipality and found that the service to these residents was between 22 to 41 
percent higher than to the general population in the municipality and ranged from a 
low of 31 persons enrolling for every 10,000 to a high of 147 in the municipality in 
which the main college site was located. We also found that the lower service rates 
for the general population in some of the municipalities were to an extent justified: 
the residents had already achieved a higher level of education and were less in need of 
a community college education. 

Race/Ethnicity: 

Service to the residents categorized on the basis of race/ethnicity varied 
widely from a low of 30 of every 1 0,000 white residents enrolled to a high of 787 of 
every 1 0,000 of those who did not identify with one of the four predominant racial 
and/or ethnic categorizations. 



Service Rates* Fall 1994 for All Residents 18 Years of Age and Older by 

Race/Ethnicity 


Race/Ethnicity 


Population 1 8 Plus 


Enrollment 


Service Rates* 


Other 


1563 


123 


787 


Asian 


26,317 


463 


176 


African American 


48,585 


667 


137 


Hispanic 


134,946 


1839 


136 


White 


219,205 


663 


30 


Total County 


430,616 


3755 


87 
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Note: Due to slight variations in census data taken from different Summary Tapes, 
totals here differ from those in other tables. 



Gender: 

Women enrolled at nearly one and one-half times the rate of men. One 
hundred one of every 10,000 women were enrolled, but only 72 of every 10,000 men. 



Service Rates* Fall 1994 for All Residents 18 Years of Age and Older by Gender 


Gender 


Population 1 8 Plus 


Enrollment 


Service Rates* 


Men 


205,099 


1481 


72 


Women 


225,596 


274 


101 


Total 


430,695 


3755 


87 



Age: 

While common wisdom says that community colleges enroll slightly older 
persons than other colleges, at this college proportionately more of the younger 
persons in the county were enrolled, especially youth eighteen and nineteen years of 
age— an age when they were presumably directly out of high school. 



Service Rates* Fall 1994 for All Residents 18 Years of Age and Older by Age Groups 


Age Groups 


Population 18 Plus 


Enrollment 


Service Rates* 


18-19 Years of Age 


14,495 


552 


381 


20-24 Years 


44,094 


1355 


307 


25-29 Years 


60,820 


605 


99 


30-49 Years 


164,248 


1132 


69 


50 and over 


147,038 


111 


8 


Total 


430,695 


3755 


87 



Multiple Characteristics: 

When we looked at multiple factors, we found that for every age group we 
were serving women at a higher rate than men, but in one municipality we were 
actually serving men at a higher rate than women— in fact at the highest rate of any 
group.. We learned that, while for most municipalities the college was serving the 
18-19 year old students at the highest rate, in one area we were serving those slightly 



45 



older at the highest rate. We found that we were serving the different racial/ethnic 
groups differently depending upon which municipality we were considering: for 
example, the service to African Americans ranged from 0 to 206 enrolled per 1 0,000 
residents depending upon the municipality; for Hispanics it ranged from 55 to 188; for 
Asians it ranged from 45 to 199; and for whites it ranged from 1 1 to 41 . 

Generally we found that we were serving young women in municipalities near 
the main college sites at the highest rate (enrolling over 400 of every 1 0,000). But in 
two municipalities the college was also serving men 18 to 19 years old at these high 
rates (494, 597). In each municipality the college was serving those 50 years old and 
older at the lowest rate (less than 20 of every 10,000 enrolled), but in the 
municipalities furthest from the main college sites, men 25 to 49 years of age were 
served at the lowest rates. 



Summary 

We knew that the college enrolled more women than men; however, before we 
conducted the study, we did not know just how differently the college was enrolling 
men and women. A college profile would have indicated that Hispanics composed 
the largest racial/ethnic category of students at the college, but after the study we 
knew better how we were serving each group within each municipality. The flat 
profile reflected the make-up of the college and was, of course, useful for 
programming, but it was not useful for understanding the data, nor for the evaluation 
of nor planning for enrollment efforts. 

However, the study was not without limitations. For example, comparable 
information, as of achievement level, for the students and of the residents was not 
always easily accessible; yet it is necessary if the analysis is to be credible. Further 
comparisons considering the achievement level of the students to that of county 
residents would be very productive but comparable data for this variable is not readily 
accessible. Also, annual rather than ten-year, census data would have made the 
tracking of the college’s progress more accurate. 

While there is a trove of data now available and awaiting further analysis, 
college-to-similar-college comparisons are needed, and questions of why it is that 
certain groups of persons are not enrolling need to be answered, college officials have 
a better understanding of who it is that the college is serving and the use of Service 
Rates has provided a vehicle for making the data accessible to the college community. 



Author’s Note: The author is appreciative of the support of Mark Oromaner, Dean of 
Planning and Institutional Research, Hudson County Community College, NJ in the 
conceptualization of this research and for the challenge to derive a means for 
expressing its service to the community in terms of its context which Marvin 
Greenberg, formerly of Rutgers University, gave the college. 
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Abstract - This study uses categorical modeling to examine gender and 
race bias in rank assignments at 12 State University of New York 
(SUNY) institutions: four universities, four colleges and four two-year 
institutions. At two-year colleges there is a pattern suggesting that 
reaching the tenured rank of Associate Professor is a glass ceiling for 
women. The four-year college pattern is diverse but with evidence of a 
sticky floor pattern for women and minorities in the non-tenure track 
rank of Lecturer. There are hurdles at most levels for three of the four 
universities. But University C results suggest much less bias than the 
other SUNY universities, indicating that bias can be avoided. 

Introduction 

Rank assignments affect the professional lives of all faculty. If gender or race bias 
enters into rank assignments, then some faculty will be in lower ranks than they 
qualify for and deserve. This is critical to an individual’s career because status and 
salary are tied to an individual’s rank. If a qualified individual is in a lower than 
deserved rank, s/he will not only have lower status, s/he will also have a lower salary. 
Thus, sticky floors - being stuck in low level ranks - and glass ceilings - not being 
able to attain the highest rank levels - can seriously impinge upon a person’s career. 
Categorical analyses can be used to diagnose where gender/race bias may be creating 
a sticky floor or glass ceiling obstructing the promotion of women and minorities. 

There is ample nationwide evidence that women and minorities are 
disproportionately at lower ranks (Gray 1988,1990,1993). This pattern is abundantly 
evident at SUNY schools as indicated by Figure 1. “Yes, but” the response frequently 
comes back, “women and minorities are in lower ranks because they tend to have less 
education or fewer years of experience or publish less.” The research reported here 
does not control for performance measures like publishing and teaching, but it does 
test whether or not education or years of experience can account for the rank 
differences found in Figure 1. 
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Figure 1 

Full-Time Faculty at 12 SUNY Schools* by Rank, Gender and Minority 
Status 
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*inclu(Tes 4 University Centers, 4 Four-Year Colleges, and 4 Two-Year 
Colleges. 

The research reported here was conducted as part of a larger project to create a 
guide book to assist higher education institutions and bargaining units to assess 
gender aind race inequities in faculty salaries, Pay Checks: A Guide to Achieving 
Salary Equity in Higher Education. 

One component of Pay Checks was a method of assessing whether or not bias 
exists in the awarding of rank. Salary equity studies have spawned a debate over 
whether or not to include rank as a predictor of salary. Some feel that rank is a 
critical element in the salary analysis (Fogel, 1986; Schrank, 1988; Moore, 1993), 
while others oppose including rank as a predictor variable (Scott, 1977; Allen, 1984). 
Those who are against including rank argue that it is a potentially tainted variable, and 
its inclusion in the regression model will underestimate the gender/race bias in faculty 
salaries (Ramsom & Megdal, 1989; Scott, 1977 AAUP Salary Kit). Since the same 
administrative units, processes amd individuals who set salaries also heavily influence 
rank amd tenure decisions, one would expect that if salary patterns are biased then 
promotion patterns are likely to be biased as well 
(Allen, 1984). 

The methods used to assess whether or not rank would be a bias-masking variable 
in predicting salary cam also be used diagnostically to suggest whether rank 
assignments incorporate barriers to the advancement of women and minorities. 




Categorical modeling can make visible the ranks where women and minorities are 
underrepresented relative to their years of experience and educational attainment. 

Methods 

The Model - Rank is a categorical variable and, therefore, can not be studied in the 
same way that continuous variables, like salary, can be studied. A categorical 
variable classifies subjects into a limited number of categories. Even if we assign 
numbers to each category or level, the categorical variable does not become 
continuous because the intervals between each level are not equal. Rank provides a 
good example. We do not know if the difference between being an Instructor and an 
Assistant Professor is the same as the difference between being an Associate 
Professor and a Full Professor, or whether an Associate Professor is worth twice as 
much as an Instructor and four times as much as a Lecturer. 

To study the categorical variable of Current Rank requires a statistical method 
called multinomial logit modeling (Aldrich & Nelson, 1984; DeMaris, 1992; Jobson, 
1992; Menard, 1995; Santner & Duffy, 1989). We call it categorical modeling for 
simplicity. Like regression modeling, categorical modeling has a dependent variable 
and predictor variables and can control all other predictor variables while it tests the 
effect of Gender or Race. In this study. Years of Experience and Highest Degree are 
held constant to see if men and women/minorities with the same qualifications are 
proportionally found in the same rank. 

We used a generalized multinomial logistic regression model, which uses 
maximum likelihood tables. A more specific logistic regression was not used because 
in ten out of twelve data sets, the levels of the response variable (Rank) were not 
parallel with regard to predictor variable effects (e.g. Years at the Institution is not 
worth the same for Instructors as it is for Leading Professors). The weighted least 
squares method of estimation was not used because it is more sensitive to very small 
data pools than the maximum likelihood tables method. Even with over 800 faculty 
(as was the case with our largest school), when separated by rank, and within rank by 
individual profiles for each of the predictor variables, the results are very small data 
pools from which the model produces estimates. Discriminant Analysis would not 
work because some of the predictor variables, such as Gender and Race, are 
categorical. 

Unlike regression, categorical modeling cannot handle many predictor variables. 
Therefore, as a rule of thumb, we reduced the number of variables analyzed whenever 
possible. 

Categorical modeling results are odds ratios, where the odds of women and/or 
minorities being in one rank rather than another rank are compared with the same 
odds for White men. Figure 2 illustrates the odds ratio, which can also be thought of 
as a pairwise comparison of adjacent ranks. For example, if there are 1 8 White- 
female Lecturers and 54 White-female Assistant Professors and 9 White-male 
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Lecturers and 54 White-male Assistant Professors, then the odds ratio for the females 
is 2 to 6, but 1 to 6 for the males. Thus, the White females are twice as likely as the 
White males to be in the rank of Lecturer as opposed to Assistant Professor. 



Figure 2 — A pairwise comparison for gender bias 
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Categorical modeling is more complicated than these simple ratios because it 
controls for the independent variables, such as Previous Experience, Educational 
Attainment and Years at the Institution, in developing the ratios. Also, to produce the 
best estimates, the model uses all available information, even for those ranks not 
directly involved in the particular comparison under review (all the data for 
Instructors, Lecturers, Full and Leading Professors are used to create estimates 
comparing Assistant to Associate Professor). 

It should be noted that these analyses do not test bias in promotion across time. 
Individuals are not traced as they move from one rank to another. These analyses 
look only at one moment in time. They assess the relative odds of being in one rank 
or the other. They answer the question: Are White men with a specific level of 
Educational Attainment and Years of Experience more likely to be at a higher rank 
than women and minorities with the same level of Educational Attainment and Years 
of Experience? Because the variables entered in these analyses are only demographic 
variables concerning education and experience, the results should be interpreted 
with caution. 

Data Sources - Twelve SUNY schools were individually analyzed for gender and 
race bias in Current Rank. These included four two-year colleges, four four-year 
colleges and four universities. The four universities in these analyses are doctoral- 
level institutions. Three of the colleges are comprehensive institutions, and the other 
college is a general baccalaureate institution. The remaining four institutions are two- 
year technical colleges. 

The data sets for these analyses were collected in 1 992 as a result of a collectively 
bargained agreement between UUP and SUNY to jointly conducted multiple regression 
analyses of faculty salaries at 29 SUNY higher education institutions. The data 
collection process involved three levels: each of the local 29 campus Employee 
Relations and/or Personnel Offices supplied the initial data information; the SUNY 
Central Administration Office of Employee Relations and Personnel Operations 
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coordinated the collection of the data; and the UUP Research Department did the 
checking and cleaning of the data sets. Not surprisingly, there were initially many data 
errors and missing data. We sent repeated reports noting data problems to each campus. 
In all cases, we persisted until we had complete data that appeared, on its face, to be 
accurate. 

In selecting the twelve SUNY institutions for the guidebook, our first criterion was 
attaining the broadest possible cross section of different sized data sets within each 
institutional type. A second consideration was the accuracy of the data and ease of 
correcting any belatedly discovered data errors. The four selected two-year colleges 
include both the largest (244) and smallest (99). The four selected four-year colleges 
range in size from 421 to 1 17 faculty members, and the four university centers, from 
81 1 to 477. This variety in faculty size and institution type enabled us to observe the 
effects of the different statistical approaches relative to this diversity. 

Variables - The dependent variable, known as the response variable in categorical 
modeling, was Current Rank. We used all the categories of Current Rank at a given 
institution', unless the modeling failed as a result of ranks with fewer than three 
individuals. This situation existed at four of our institutions. One university and one 
college had only two instructors, so instructors were dropped from these analyses. 

Two colleges each had only one leading professor so leading professors were dropped 
from these analyses. 

We included the predictor variables Race, Gender, Highest Degree, Years Since 
Highest Degree at Time of Hire,^ Previous Experience at the Time of Hire,Years at 
the Institution, and quadratic terms to control for curvilinearity in the time related 
variables.^ Bearing in mind that it is important to keep the number of variables down, 
we included the available variables that most influence achieving rank. 

To avoid zero cells, we did not include every race/gender combination (White 
male. White female, African American male, African American female, Asian male, 
Asian female. Latino male. Latino female) in the model.'* Instead, we analyzed White 



’ We found it important to drop from the analysis any empty rank categories. If you do not do this and - 
mistakenly review the results with the impression that the rank was included, you may misinterpret the 
output. So, before starting any modeling, we identified the ranks with faculty in them and dropped all 
others. 

^This variable is a specific, verifiable date, but it may credit women with more experience than they have 
earned. So, to be conservative, we chose to include the variable previous experience as well. The previous 
experience data is less reliable than the Years Since Degree at Time of Hire due to the difference in 
perceptions regarding what constitutes valid previous experience. However, we included it to minimize 
any tendency to over credit women for experience relative to Years Since Degree. 

^To eliminate the potential of redundancy when using quadratic terms we centered the time variables. 

"*See Appendix B of Haignere et. al. 1996 for a discussion of zero cells and the problems they create in 
modeling categorical data such as rank. 
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females, Minority males and Minority females as compared to White males. 
Likewise, though we had nine levels of educational attainment, we found that at 
SUNY universities and four-year colleges, 90 percent or more of the faculty have 
either a master’s or doctoral degree. As a result, we reduced the variable to two 
levels: Ph.D. and non-Ph.D. At SUNY two-year colleges, most faculty have masters 
degrees. Thus, we used MA & Above and Below MA in the categorical modeling 
analyses. 



Results 

Figure 3 provides the categorical modeling results for the 12 SUNY schools. At 9 
of the 12 SUNY institutions examined there is clear evidence that women and 
minorities are not assigned to ranks as high as White men with the same highest 
degrees and comparable years of experience. 

As a guide in interpreting the tables in Figure 3, University A rank assignments of 
Lecturer as opposed to Assistant Professor can be read as follows: The odds ratio for 
the category White female is 2.96^. This means that White women are 2.96 times 
more likely than White men to be Lecturers rather than Assistant Professors. When 
reviewing the same rank assignments for Minority males, the odds ratio is 3.69. This 
means that Minority males are 3.69 times more likely than White males to be 
Lecturers rather than Assistant Professors. Minority females come the closest to 
approximating the distribution of White men in these two ranks. They are 1 .75 times 
more likely to be Lectures rather than Assistant Professors. 

University Centers - At the University Centers, women and minorities were likely to 
be in lower ranks than White men with comparable career attributes. At University 
A, for instance, the findings suggest gender bias hurdles for each rank assignment 
comparison and both gender and race bias at the lowest comparison. Lecturer to 
Assistant. In contrast. University C results suggest little gender and race bias. 
Minority females at University C are shown to be 2.92 times more likely to be 
Lecturers than Assistant Professors. However, this result should be interpreted in 
light of the low numbers of Minority women 

Universities B and D findings suggest bias patterns that fall between the patterns 
observed at Universities A and C. At universities B and D there is evidence of both 
gender and race bias between the rank assignments of Assistant and Associate. 
University B results suggest more gender bias at the lower rank comparisons, while 
University D results suggest more gender bias at the higher rank levels. 



’ The p-Value for each ratio indicates the statistical significance levelthat would be used if this were a 
random sample. However, we are studyingthe entire population of faculty at each institution in 1992. The 
Whitefemale ratio for Lecturers to Assistants at University A reaches a p-Valueof.7 which is just short of 
statistical significance at the .05 level. Formore information on the significance of significance when you 
are studyinga population see Haignere et. al. 1996 p. 75-77. 
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Figure 3 - Bias in Rank Results at 12 SUNY Schools 
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Figure 3 (Continued) 
University D 
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Figure 3 (Continued) 
College G 
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Figure 3 (Continued) 
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White Male 






0.12 


0.0748 


0.40 


0.315 


Minority Female 
White Male 






1.93 


0.6916 


0.26 


0.377 






# Lect. 
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# Full 






Wh Mala 


0 


28 


47 


59 




Wh. Fern. 


0 


32 


22 


13 




Min. Male 


0 


3 


4 


_6 




Min. Fern. 


0 


4 


1 


1 


Totals^ 


0 


67 


74 


79 
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Four- Year Colleges -The variations between the four-year colleges suggest no 
consistent pattern except that women and minorities are much more likely to be non- 
tenure track Lecturers than are White men with comparable career attributes. College 
F results suggest gender bias also at the highest rank assignments of Associate and 
Full Professors. The results for the other three colleges suggest gender bias is most 
evident in assignments below the Full Professor rank. Colleges E and H display a 
pattern similar to the two-year colleges, a gender hurdle in reaching the rank of 
Associate Professor. The numbers of minorities in the rank comparisons at the four- 
year colleges are too small to form the basis of conclusions. However, it appears that 
there may be a race bias hurdle between Assistant and Associate rank assignments at 
three of these four colleges. 

Two-Year Colleges - With the exception of Technical Institution L, there is a notable 
lack of women and minorities at the two-year institutions examined. However, the 
numbers are large enough to suggest diagnostic conclusions concerning gender bias 
across the Assistant to Associate ranks. Women have much lower odds than White 
men of being awarded the rank of Associate Professor. If they manage to become 
Associate Professors, their odds of becoming Full Professors are similar to White 
males. Thus, getting into the Associate Professor rank appears to be a glass ceiling for 
women at 3 two-year campuses. Technical Institution J, the smallest school, is the 
exception, and shows no bias toward women. There are not enough minorities at these 
schools to support any conclusions about race bias. 

. Conclusions 

Three of the twelve schools we studied show very little bias in Current Rank, one 
at each institutional type. University C, College G and Technical Institution J. This 
suggests that gender/race bias in rank assignment processes is not universal and some 
schools have developed more equitable processes than others. At the SUNY 
institutions where our analyses indicate bias in rank assignment, the results could be 
used to point to where gender/race bias may be creating a glass ceiling blocking the 
promotion of women and minorities. 

Concerning minorities, the findings indicate a sticky-floor phenomenon holding 
minorities in the non-tenure track rank of Lecturer. Race bias hurdles reaching the 
Associate Professor rank are also suggested at some schools. Combining all 
minorities, as we have done, may mask bias when there is substantial bias against one 
minority type, but not the others. Despite this potential problem, we chose to 
combine categories in order to avoid zero cells and to produce better estimates by 
creating larger pools of data. Even so, as we indicated above, the numbers of 
minorities are often too small to support diagnostic conclusions. 

As we have cautioned above, these analyses do not test bias in promotion across 
time. Individuals are not traced as they move from one rank to another. These 
analyses look only at one moment in time. They assess the relative odds of being in 
one rank or the other. They answer the question: Are White men with a specific level 
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of Educational Attainment and Years of Experience more likely to be at a higher rank 
than women and minorities with the same level of Educational Attainment and Years 
of Experience? Because the variables entered in these analyses are only demographic 
variables concerning education and experience, the results should be interpreted 
with caution. They indicate whether Current Rank appears to be biased based on only 
a few predictor variables which include no discipline or performance information. 

The results are best used only diagnostically to flag where bias may be creating a 
sticky floor or a glass ceiling blocking the promotion of women and minorities. 

Further assessments of the institutional processes involved would be necessary to 
indicate whether or not biases really exist. 
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INTRODUCTION 

This paper is intended as an introduction to the recent National Research Council’s 
(NRC) report on doctoral education in the United States. Particular attention will be 
given to: placing the NRC study in the context of earlier studies of doctoral 
education; briefly discussing how the NRC data were obtained, examined and 
presented; and outlining some of the ways the NRC data can be used to better 
understand doctoral education at Rutgers. In addition, some of the basic findings of 
the NRC report as they relate to Rutgers University will be presented. 

Interest within and outside the academy in the quality of doctoral education in the 
United States has been heightened with the release last Fall of NRC’s Research- 

Doctorate Programs in the United States: Continuity and Change. ^ This publication 
updates an earlier study by the Conference Board of Associated Research Councils 
(CBARC), An Assessment of Research-Doctorate Programs in the United States, 
which was published as a five-volume report in 1982. The NRC publication has 
received much attention in the media, with articles appearing in the Chronicle of 
Higher Education, Change magazine and the New York Times. Some reasons for 
interest in the NRC report include: 

• its size - the NRC study surveyed 3,634 doctoral programs at 274 American 
universities in 41 different academic fields; 

• its reputational rankings - subjective program and faculty quality 
assessments are provided by knowledgeable academics rather than magazine editors 
or media pundits; 

• its wealth of data - both reputational and objective measures on doctoral 
programs, as well as institutional-level data, are included in this study; 
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• its consistency with the 1982 study - many of the measures used in the NRC 
report can be compared with results of the earlier CBARC study; 

• its presentational structure - data for institutions and their respective 
programs were rank-ordered by various measures found in the report, thereby giving 
readers a quick comparative interpretation of the data; 

• its accessibility - much of the data used in the NRC report are not only 
available in printable form, but can also be accessed electronically from NRC’s 
World Wide Web site; and 

• its potential for further analysis and research - researchers have the 
capability of further analyzing the data found in the NRC study beyond that 
undertaken by the authors of the report. 



BACKGROUND TO THE NRC STUDY 

The concern for assessing the quality of doctoral education in the United States has 
been long-standing. The first serious effort to assess doctoral programs in the 
United States was undertaken by Raymond Hughes in 1925. Hughes, the then 
president of the University of Miami at Ohio, conducted a survey of doctoral 
programs in twenty fields, representing thirty-eight of sixty-seven institutions that 
offered the Ph.D. at that time. Hughes compiled a list of well-noted scholars and 
asked each scholar to evaluate between twenty and sixty programs apiece in the 
scholar’s respective academic field. 

Although the work of Hughes was well known within academia and did not go 

uncriticized,2 it wasn’t until the 1960s and especially the 1970s that studies 
designed to assess the quality of doctoral education in the United States expanded. 
Reviewers have categorized the many and varied studies that have sought to assess 
the quality of doctoral programs into three general types: reputational studies; 

objective indicator studies; and quantitative correlate studies. ^ Most of the well- 
known studies have been of the reputational type — the subjective evaluation of 
doctoral programs by faculty and/or academic administrators (e.g., departmental 
heads and deans) to arrive at a rating for programs within an academic field. 
Although well-known, these studies have also received the most criticism. The 
essential argument against reputational studies has been that these inquiries are 
more an assessment of a program’s faculty reputation rather than an indicator of a 
program ’ s o veral 1 qual ity . 

One set of studies has sought to correct this limitation by assessing a program’s 
quality through readily available objective indices of faculty research production. 
The rationale for this approach is that the quality of a department is necessarily 
dependent on the quality of its faculty. However, the problem with this position 
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relates to the lack of consensus with how faculty quality can be measured. Such 
studies have relied, for the most part, on measures such as the number and type of 
publications and citations, but have also included non-faculty indices such as 
student outcomes and institutional or departmental financial resources. 

Another type of study, often called quantitative correlate studies, has aimed at 
understanding the components of doctoral program quality by seeking to determine 
those variables that are most strongly related to reputational information about a 
program. Using many of the measures found in objective indicator studies, 
correlate studies have the potential of contributing to institutional plarming by 
identifying measures that underlie quality of different doctoral programs (e.g., 

faculty size, number of full-time professors, etc.).^ 

Many criticisms, both methodological and conceptual, have been leveled against all 
three types of studies. In addition to the criticism that many of these studies, and 
especially reputational studies, tend to confuse the reputation of a doctoral program 
for quality, other criticisms that have been directed at studies of doctoral education 
include the existence of: rater bias; outdated data; unreliable and incorrect data; 
unidimensionality (i.e., the lack of including more than one indicator of quality); 
and an atheoretical focus (this latter criticism is often cited against correlate 
studies). Although these critiques tell us that we should be careful when 
interpreting the results of these studies, they also have contributed to an ongoing 
refinement of studies of this kind. The NRC study represents the most recent and 
perhaps the most comprehensive response to date to the shortcomings of the earlier 
studies. 



THE METHODOLOGY OF THE NRC STUDY 

Average characteristics of the institutions participating in the NRC report are found 
in Table 1. This table distinguishes among public and private institutions and 
various categories of the Carnegie classification system of higher education 

institutions.^ Table 1 displays data derived from 105 private and 169 public 
institutions. Research I institutions comprise the category with the most schools 
(there are 90 Research I institutions accounting for 33% of all Softools included in 
the NRC study). 

The main purpose of the NRC report was to present figures on individual programs 
in 41 academic fields. Academic fields included in the NRC report were based on 
three criteria: 1) the number of Ph.D.s produced nationally; 2) the number of 
programs training Ph.D.s within a particular field; and 3) the average number of 
Ph.D.s produced per program. Overall, 3,634 research-doctorate programs at 274 
universities were included in the 1993 study. 
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Two types of measures were collected for this study: reputational measures and 
data from secondary sources. Reputational measures were obtained for each 
institutional program based on the ratings given by faculty within a particular field. 
For each program, ratings of quality (of the faculty), effectiveness (of educating 
scholars), and the perceived change in quality (over the last five years) were 
obtained. Each faculty member who was included in the study was asked to rate 
approximately 50 programs in his or her field. For each institution, raters were 
given a faculty listing to be used as a reference. Approximately 100 ratings per 
program were obtained and the mean of these ratings was used to obtain an overall 

score for each program. ^ 

The data from secoridary sources include information from four categories: faculty 
(e.g., total number of faculty in program, % of faculty publishing between 1988 and 
1992); students (e.g., % of female students); doctoral recipients (e.g., % of doctoral 
recipients with research or teaching assistantships as a primary source of support, 
median number of years to complete Ph.D.); and institution (e.g., year of first Ph.D., 
average research expenditure). 

One of the features of the NRC report that makes it so impressive is its spanning of 
the three categories of studies mentioned above. Not only are reputational and 
objective measures of doctoral programs presented, additional analysis and 
discussion are also provided with regard to the relationship between the reputational 
measures used in the study and some of the secondary indices compiled for 
individual doctoral programs. The NRC report also presents some findings 
comparing institutions with regard to certain measures of change obtained from the 
1982 CBARC and 1993 NRC studies (e.g., the change in the “Scholarly Quality of 
Program Faculty”) and closer examination of the items that profile graduates of the 
various doctoral programs included in the report. 



IMPORTANCE OF THE NRC RANKINGS TO RUTGERS 

NRC data can be useful to faculty and staff at Rutgers because they place our 
doctoral programs in a multi-dimensional comparative context. First, these data 
provide longitudinal information. For example, the designers of the NRC study 
insisted on the comparability of as many measures as possible between the 1982 
CBARC and 1993 NRC reports. Thus it is possible to see how a particular program 
has fared during the period between the administration of the two studies. Although 
not an overt longitudinal measure, the reputational measure, “Change in Program 
Quality in Last Five Years,” provides additional insight with regard to program 
status over the recent past in the eyes of peer faculty. 

Second, these data allow us to compare Rutgers ’ program characteristics to those 
of similar programs offered by other universities. These comparisons can be made 
against national averages or they can be made against specific subgroups of 



er|c 



65 



universities. The importance of such peer comparisons cannot be over stated. 
Institutions of higher education vary by size, type of control, educational mission 
and other characteristics (e.g., regional distinctions and physical location) that often 
make comparisons among the different groupings of institutions senseless and, at 

times, misleading.^ Even among members of the Association of American 
Universities (AAU), of which Rutgers is a member, a further distinction needs to be 
made between private and public AAU schools when deciphering the data in the 
NRC report. For example. Figure 1 shows that private AAU institutions 
consistently have more programs listed in the top ten of the “Scholarly Quality of 
Program Faculty” measure, on average, than public AAU institutions. Although the 
extent of the difference between these averages varies by academic area, it is clear 
from Figure 1 that doctoral programs from private AAU institutions are rated as 
having a higher quality of faculty than public AAU institutions. Some would 
interpret the data in Figure 1 to be an indication of the superiority of private AAU 
institutions in the administration of doctoral programs, while others would argue 

that these rankings instead show the biased nature of such reputational rankings.^ 
However, an equally valid interpretation would acknowledge the success of doctoral 
education among the private AAU institutions while recognizing the confluence of 
historical and financial factors that make private AAU institutions different from 
all other institutions of higher education, including public AAU schools. 

Third, these data allow us to analyze Rutgers’ academic achievement at a variety of 
levels. For example, academic success can be evaluated for specific programs (e.g.. 
Psychology, Aerospace Engineering) and academic areas (e.g.. Arts and 
Humanities, Biological Sciences). In addition, overall numbers for Rutgers (across 
academic areas) can be derived and assessed. 



SELECTED FINDINGS 

Quality of Faculty Ratings (Table 2a, attached) 

Comparison Between Rutgers ’ 1982 and 1993 Ratings 

Quality of faculty ratings increased for most Rutgers’ doctoral programs between 
1982 and 1993 for which there are comparable data. The most notable increases 
occurred in Philosophy (from a mean rating of 2.5 on a six-point scale in 1982 to 
3.8 in 1993), Chemical Engineering (1.8 to 2.7), Electrical Engineering (1.8 to 2.8), 

and Computer Science (2.4 to 3.3).^ 

Comparison of Rutgers ' 1993 Ratings with National and Peer Statistics 

In general, Rutgers’ quality of faculty ratings are comparable to or higher than the 
national average. The following Rutgers’ doctoral programs received ratings 
substantially above the national average: English (mean rating of 3.9 for Rutgers vs. 
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the national average of 2.7), Philosophy (3.8 vs. 2.8), Biochemistry and Molecular 
Biology (3.5 vs. 2.6), Mathematics (4.0 vs. 2.8), Physics (3.8 vs. 2.8), History (3.7 
vs. 2.8) and Psychology (3.8 vs. 2.7). In general, Rutgers received quality of faculty 
ratings which are comparable to that of the public AAU averages, and the public 
AAU schools scored consistently higher than the national average. Moreover, 
Rutgers does very well when compared to the average scores for public AAU 

institutions from the Northeast region. ^ ^ 

Change in Program Quality (Table 2b, attached) 

Consistent with the findings discussed above, the quality of the faculty in almost all 
Rutgers’ programs is considered to have significantly improved during the five 
years prior to 1993. On a scale from -1.0 to +1.0, where 0 represents no change, 
almost all Rutgers’ programs received positive ratings between .23 and .64. In 
comparison, the national average positive ratings for programs on this measure 
generally range from about .01 to .24. Rutgers also scores favorably well when 
national and Northeast public AAU institutions are considered. Average positive 
ratings for the various doctoral programs among national public AAU schools range 
from .02 to .42, among the Northeast public AAU schools these scores range from 
0.0 to .52. 

Program Effectiveness Ratings (Table 2c, attached) 

Comparison Between Rutgers ’ 1982 and 1993 Ratings 

Program effectiveness ratings increased for many Rutgers’ programs between 1982 
and 1993 for which there are comparable data. These increases ranged from .1 in 
Physics and French Languages and Literature to 1.2 in Philosophy and Chemical 
Engineering. 

Comparison of Rutgers ’ 1993 Ratings with National and Peer Statistics 

In general, Rutgers’ program effectiveness ratings are comparable to or higher than 
the national average. The following Rutgers’ programs received ratings 
substantially above the national average: English (mean rating of 3.7 for Rutgers vs. 
the national average of 2.8), Mathematics (3.6 vs. 2.6), History (3.7 vs. 2.8) and 
Psychology (3.7 vs. 2.8). In general, Rutgers received program effectiveness ratings 
which are comparable to those of national public AAUs and exceeded those of the 
Northeast public AAUs. 

Faculty and Student Information (Table 3, attached) 

Rutgers’ faculty and student statistics are similar to both public AAU and the 
national average on a number of measures. For example, the rainge for the 
percentage of full professors among various university groupings listed in Table 3 is 
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between 49.9% and 55.5%, with 53.9% of Rutgers’ full professors participating in 
doctoral programs. All four categories of institutions in Table 3 also have an 
approximate percentage of faculty members who received an honor or award 
between 1986 and 1992 (these values range from 8% for public AAU schools in the 
Northeast to 9.7% for national and public AAU schools). Furthermore, on average, 
each Rutgers’ program produced approximately 30 Ph.D.s between 1988 and 1992, 
compared to 29 on the national level and among Northeast public AAU schools. 
Public AAU schools averaged a slightly higher number of Ph.D.s per program 
during this period (36.6). On average, it took slightly under ten years (9.9) for 
students to complete their Ph.D. at Rutgers; this average compares to 9, 9.5 and 9.2 
for national. Northeast public AAU and national public AAU schools, respectively. 

Rutgers is somewhat different than the national average on three other measures. 
First, given Rutgers’ size, it should be no surprise that the average number of 
faculty participating per program at Rutgers (37.5) is significantly greater than the 
national average (21.0) and somewhat greater than the public AAU (30.6) average. 
Second, the percentage of program faculty at Rutgers publishing between 1988 and 
1992 (76.8%) was somewhat greater than the national average (69.7%) and on par 
with the public AAU average (78.2%). The percentage of Ph.D.s awarded to 
women is higher at Rutgers (36.1%) than the national (30.7%) or public AAU 
average (30.4%). Finally, Rutgers also has a higher percentage of Ph.D.s awarded 
to minorities when compared to all public AAU and Northeast public AAU 
institutions (6% compared to 5.5% and 5.8%, respectively). 
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ENDNOTES 



1 While the NRC survey was only released last September, the report is based on 
survey data collected during 1993 and secondary data from 1986 through 1992. 

2 Two frequently mentioned criticisms of Hughes’ work were that the size of his 
panel of experts for rating a given program was too small and that there was a 
geographical imbalance among the scholars evaluating the programs, with most of 
the raters residing in the Northeast and Midwest. 

^ Conrad and Blackburn (1985) and Tan (1986). 

4 Ehrenberg and Hurst (1996) is a recent example of how the combination of 
reputational measures and objective indicators may be used to inform institutional 
planning. 

5 All American colleges and universities that are degree-granting and accredited by 
an agency recognized by the U.S. Department of Education are included in the 
Carnegie classification system. There are twenty classification categories in the 
Carnegie system, with the bulk of doctoral degrees coming from four categories. 
These include Research I, Research II, Doctoral I and Doctoral II institutions. For a 
full description of the various Carnegie classifications, see The Chronicle of Higher 
Education, April 6, 1994. For a complete listing of AAU institutions, both public 
and private, see the 1995-1996 Rutgers Fact Book, p. 107. 

^ More specifically, these averages do not include the two highest and lowest 
scores in the computation of the mean. 

2 One misleading effect of comparing institutions of differing sizes, types and/or 
mission is what the authors of one report (Graham, A.G., R.W. Lyman, and M. 
Trow, 1995) have called epistemic drift — the tendency for institutions to respond to 
external accountability requirements and reports in a manner that may be contrary to 
their own interests. For example, an institution with a few doctoral programs but a 
strong undergraduate program may seek to aspire to the rating levels that major 
private universities often achieve in reports such as the NRC study by re-allocating 
its energies and resources from the area that is its strength (i.e., undergraduate 
instruction). The likelihood that such an institution will achieve significantly better 
ratings for its few doctoral programs is low, given that it probably lacks the 
institutional resources needed for such improvement, while at the same time 
weakening the area where it is quite successful. 

8 This is akin to the “halo” effect argument that sees reputational rankings 
contributing to a pecking order among institutions such that evaluations of a specific 



|ee|c 



67 



program are inextricably connected to the institution’s overall reputation. As Tan 
has. noted (1986, p. 231), some critics see this as possibly contributing to an 
institution’s lack of innovation due to the belief that, such embracing of change 
imperils that institution’s reputational ranking. In certain ways, this is the opposite 
of the tendency toward epistemic drift noted above. 

9 Although this comparison of reputational scores between the 1982 and 1993 
reports indicate that many Rutgers’ doctoral programs experienced gains in their 
ratings, it is not entirely clear if these increases are the result of “real” improvement 
or “grade” (i.e., rating) inflation (i.e., that the 1993 raters were more lenient in their 
evaluation of programs). One indication that these increases are, in fact, reflective of 
improvement in doctoral education at Rutgers is the positive scores for Rutgers 
found in the “Change in Program Quality” reputational measure that is discussed 
shortly. For a discussion of score changes and the existence of grade inflation, see 
Appendix R in the NRC report. 

In addition to Rutgers, these Northeast public AAU institutions include 
Maryland, Penn State, SUNY at Buffalo and Pittsburgh. 
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Table 1 

{Characteristics of Institutions Part'dpating in the NRC Study by Control, Carnegie Qassification, and AAU Membership 



AVERAGE 






PUBUC 










PRIVATE 








AA 


RESI 


RES II 


DOCTI 


DOCTII OTHER* 


AA 


RESI 


RES II 


DOCTI 


DOCTII OTHER 


Total R&0(x $1,000) 


136.188 


113.064 


33,569 


8.118 


12.498 


24,225 


109.784 


112785 


23.783 


5.415 


7.581 


9.115 


Federal R&0(x $1,000 


78.009 


62,100 


13266 


2.437 


5.659 


13.408 


80.327 


83.028 


14.489 


3.499 


5.304 


8,199 


Ubr»y-1992 

Vohinw 


3,975.938 


2980,04 


1.567,100 


1.019.882 


614,78 


296,438 


3.562.326 


3,439.441 


1.508.952 


983.700 


654,222 


351.87 


Expend (X $1,000) 


17.58 


13,85 


7.139 


4.837 


3.30 


2389 


18.118 


18268 


7.58 


4,701 


3,15 


1.82 


Enrollment ' 1992 
Total 


30.81 


28.86 


20.020 


17.410 


12,64 


4.821 


12.673 


13.378 


11,10 


9,004 


5.81 


2.34 


Qraduata Student 


7.00 


6,07 


3.637 


3.690 


2,38 


919 


5.166 


5,111 


3.19 


3.142 


127 


75 


Nuifijer Of Prograns 
198 


2 


2 


1 








22 


2 


1 








199 


3 


2 


1 








26 


2 


1 








Rrst Year PtvD. Awards 


187 


187 


1891 


1934 


191 


1952 


1880 


186 


187 


1897 


188 


188 


^Includes institutions with Carnegie Qassification in specialized doctoral programs such as medicine, theology, etcx 
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A\^rage Number of Doctoral Rx^gran^ Top Teriof Qual ity of Faculty for Each Mcgor Academic Area 
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Table 2a 

Quality of Faculty Ratings 





RLTTGERS 


RUTGERS 


1993 


1993 


1993 


ACADEMIC AND PROGRAM AREAS 


1982 MEAN 


1993 MEAN 


NATIONAL 


PUBUC 


NORTHEAST 




RATING 


RATING 


STATISTICS 


AAU. 


PUBUC AAU 


NEWBRUNSWICK 












ARTS AND HUMANIT1E 












Aft History 


2. 


3. 


3. 


2. 


2. 


Coniparative Literature 


* 


2. 


3. 


2. 


2. 


English Language and Literatur 


3. 


3. 


2. 


3. 


3. 


French Language and Literatur 


2. 


3. 


3. 


2. 


2. 


German Language and Literatur 


1. 


1. 


3. 


3. 


1. 


Music 


3. 


3. 


3. 


3. 


3. 


Philosophy 


2. 


3. 


2. 


3. 


3. 


Spanish and Portuguese Language & Literatur 


2. 


2. 


3. 


3. 


2. 


BIOLOGICAL SCIENCE 












Biochemistry and Molecular Biology 


3. 


3. 


2. 


3. 


2. 


Cell and Developmental Biology 


* 


3. 


2. 


3. 


2. 


Ecology. Evolution and Behavior 


* 


3. 


2. 


3. 


3. 


Molecular and General Genetics 


* 


3. 


2. 


3. 


3. 


Neuroscience 


* 


3. 


2. 


3. 


2. 


Pharmacology 


* 


3. 


3. 


3. 


3. 


Physiology 


3. 


3. 


2. 


3. 


3. 


ENGINEERIN 












Biomedical Engineerin 


• 


3. 


3. 


3. 


3. 


Chemical Engineerin 


1. 


2. 


2. 


3. 


2. 


Electrical Engineerin 


1. 


2. 


2. 


3. 


3. 


Materials Science (Ceramic Science & Engineering) 


• 


3. 


3. 


3. 


3. 


Materials Science (Materials Science & Engineering) 


• 


2. 


3. 


3. 


3. 


Mechanical Engineerin 


2. 


3. 


2. 


3. 


3. 


PHYSICAL SCIENCES & MATHEMATIC 












Chemistry 


2. 


3. 


2. 


3. 


3. 


Computer Sdenc 


2. 


3. 


2. 


3. 


3. 


Mathematic 


3. 


4. 


2. 


3. 


3. 


Physic 


3. 


3. 


2. 


3. 


3. 


Statistics and Biostatistics 


3. 


3. 


3. 


3. 


2. 


SOaAL AND BEHAVIORAL SCIENCE 












Anthropology 


3. 


3. 


3. 


3. 


3. 


Economic 


1. 


2. 


2. 


2. 


2. 


Geography 


2. 


3. 


3. 


3. 


3. 


History 


3. 


3. 


2. 


3. 


3. 


Political Science 


3. 


3. 


2. 


3. 


2. 


Psychology 


3. 


3. 


2. 


3. 


3. 


Sociology 


3. 


3. 


2. 


3. 


2. 




RUTGERS 


RUTGERS 


1993 


400^ Dl IDI I/' 




ACADEMIC AND PROGRAM AREAS 


1982 MEAN 


1993 MEAN 


NATIONAL 


rUDUu 

DOCT II 


* 




RATING 


RATING 


STATISTICS 






NB/VARK 












BIOLOGICAL SCIENCES 












Biochemistry and Molecular Biology 


• 


1. 


2. 


1. 




Ecology. Evolution and Behavior 


• 


2. 


2. 


2. 




Neuroscience 


• 


2 


2. 


1. 




PHYSICAL SCIENCES & MATHEMATICS 












Chemistry 


1. 


1. 


2. 


1. 




SOaAL AND BEHAVIORAL SCIENCES 












Psychology 


2. 


0. 


2. 


1. 





* Programs not included in the 1982 study. 

Note: Quality of Faculty Rating s are ba sed on a s ix point scale in which "5" = Distinguished and "0*' = Not sufficient for doctoral education. 
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fable 2 

Change in Program Quality 



ACADEMIC AND PROGRAM AREAS 


RUTGERS 1993 
n/EAN RATING 


1993 NATIONAL 
STATISTICS 


1993PUBUC 

AAU 


1993 

NORTVEAST 

PUBUCAAU 


NEWBRUNSV\/ICK 
ARTS AND HUMANITIE 










Art History 


.3 


.0 


.0 


.1 


Comparative Literature 


.0 


.0 


-.0 


-.0 


English Language and Literatur 


.5 


.1 


.2 


.3 


French Language and Literatur 


.2 


.0 


.0 


■ , .1 


German Language and Literatur 


-.2 


*.o 




r.1 . 


Music 


-.4 


.0 


.0 


-.0 . 


Philosophy 


.9 


.0 


.1 


.1 


Spanish and Portuguese Language and Literature 
BIOLOGICAL SOENCE 


.0 


.0 


. .0- 


.0 ■ 


Biochemistry and Molecular Biology 


.5 


.2 


.2 . 


'.1 ■ 


Cell and De^opmental Biology 


.4 


.2 


.3 


.1 


Ecology. Evolution and Behavior 


.3 


.2 


.2 


.3 


Molecular and Gerera) Genetic 


.5 


.2 


.3 


, .5 


Neuroscience 


.6 


.2 


.2 


.1 


Pharmacology 


-.7 


.2 


.2 


.4 


Physiology 


.5 


.2 


.2 


. .3 


ENGINEERIN 










Biomedical Engineerin 


.3 


.2 


.4 


.4 . 


Chemica) Engineerin 


.4 


.0 


.1 


.2 


Bectrica) Engineerin ' 


.3 


.1 


.2 


.2 


Materials Sdenoe (Ceramic Sderx» and Engineering 


.3 


.0 ; 


.2 


.1 


Materials Sdenoe (Materials Sdenoe and Engineering 


.3 


.0 


.2 


.1 


Mechanical Engineerin 


.3 


.0 


.2 


.2 


PHYSICAL SaENCES & MATHEMATIC 










Chemistry 


.2 


.0 


.2 


.1 


Computer Sdenc 


.2 


.1 


.2 


- .1 


Mathematic 


.4 


.1 


.2 


.3 


Physic 


.6 


.1 


.2 


.2 


Statistics and Biostatistic 


.1 


.0 


.0 


.0 


SOOAL AND BEHAVIORAL SOENCE 










Anthropology 


.3 


.0 


.1 


.2 


Ecofxxnic 


.2 


.0 


.0 


.1 


Geography 


.5 


.0 


.0 


.2 


History 


.4 


.1 


.2 


.1 


Political Sderc 


.3 


.0 


.1 


.1 . 


Psychology 


.3 


.0 


.1 


.1 


Sodology 


.2 


.0 


.0 


. ■ -2 


ACADEMIC AND PROGRAM AREAS 


RIGGERS 1993 
n/EAN RATING 


1993 NATIONAL 
STATISTICS 


1993PUBUC 

DOCTII 




NEWARK 

BIOLOGICAL SOENCE 








- 


Biochemistry and Molecular Biology 


-.2 


.2 


.0 




Ecology. Evolution and Behavior 


-.3 


.2 


.0 




Neurosdence 


.7 


.2 


.0 




PHYSICAL SaENCES & MATHEMATIC 










Chemistry 


.0 


.0 


-.0 




SOOAL AND BEHAVIORAL SOENCE 










Psychology 


-.2 


.0 


.1 





Note; Ratings are based on a scale of -1 to +1 in which "1" rneans better than 5 years ago and "-1" means poorer than 5 years ago. 
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Table 2c 

Quality of Program Effectiveness Ratings 



ACADBVDC AND PROGRAM AREAS 


PSJTGEFS 
1982 MEAN 
RATING* 


RUTGERS 
1993 nSAN 
RATING 


1993 

NATIONAL 

STATISTICS 


1993 PUBUC 
AAU 


1993 

NORT>CAST 
PUBUC AAU 


NBAfBRUNS\MCK 
ARTS AND HUMATiTlE 












Art Histofy 


2. 


3. 


3. 


2. 


2. 


Comparative Uterature 


** 


2. 


3. 


2. 


Z 


English Language and Uteratur 


3. 


3. 


2. 


3. 


3. 


French Language and Uteratur 


2. 


2. 


2. 


2. 


Z 


German Language and Uteratur 


1. 


1. 


2. 


Z 


1. 


Music 


2. 


3. 


2. 


3. 


z 


Philosophy 


2. 


3. 


2. 


3. 


3. 


Spanish and Portuguese Language & Uteratur 


2. 


2. 


2. 


3. 


Z 


BIOLOGICAL SOENCE 












Biochemistry and Molecular Biology 


** 


3. 


2. 


3. 


3. 


Ceil and De^opmental Biology 


** 


3. 


2. 


3. 


2. 


Ecology. Evolution and Behavior 


** 


3. 


2. 


3. 


3. 


Molecular and General Genetics 


** 


3. 


3. 


3. 


3. 


Neuroscienoe 


** 


3. 


2. 


3. 


3. 


Pharmacology 


** 


3. 


3. 


3. 


3. 


Physiology 


3. 


3. 


3. 


3. 


3. 


ENqiNEERIN 












Biomedical Engineerin 


** 


3. 


3. 


3. 


3. 


Chemical Engineerin 


1. 


3. 


2. 


3. 


3. 


Bectrical Engineerin 


1. 


2 


2. 


3. 


3. 


Materials Sdenoe (Ceramic Sdenoe & Engineering 


** 


3. 


3. 


3. 


2. 


Materials Sdenoe (Materials Sdenoe & Engineering 


** 


2. 


3. 


3. 


Z 


Mechanical Engineerin 


2. 


3. 


2. 


3. 


3. 


PHYSICAL SOENCES & MATHEMATIC 












Chemistry 


2. 


2. 


2. 


3. 


3. 


Computer Sdenoe 


2. 


3. 


2. 


3. 


2. 


Mathematics 


3. 


3. 


2. 


3. 


3. 


Physic 


3. 


3. 


2. 


3. 


3. 


St^stics artd Biostatistics 


Z 


3. 


2. 


3. 


2. . 


SOaAL AND BEHAVIORAL SQENCE 












Anthropology 


2. 


3. 


2. 


3. 


3. 


Economic 


1. 


2. 


2. 


2. 


2. 


Geography 


2 


3. 


3. 


3. 


3. 


History 


3. 


3. 


2. 


3. 


3. 


Political Sderce 


2. 


3. 


2. 


3. 


2. 


Psychology 


3. 


3. 


2. 


3. 


3. 


Soddogy 


2. 


2. 


2. 


3. 


2. 


ACADEMIC AND PROGRAM AREAS 


RUTGERS 

1982IVEAN 

RATING* 


RUTGERS 

1993IVEAN 

RATING 


1993 

NATIONAL 

STATISTICS 


1993 PIBUC 
DOCTII 




NEmRK 

BIOLOGICAL SaENCES 












Biochemistry ard Molecular Biology 


** 


1. 


2. 


1. 




Ecology, Evolution and Behavior 


** 


3. 


2. 


2. 




Neurosderxa 


** 


3. 


2. 


1. 




PHYSICAL SOENCES & MATHEMATICS 
Chemistry 


1. 


2. 


2. 


1. 




SOOAL AND BEHAVIORAL SOENCES 












Psychology 


Z 


0. 


2. 


2. 





! * Recalculated 1 982 scores from the four-point to the six-point scaling system. 

I** Programs not included in the 1982 study. 

; l^e: Prog^ Bfectiveness Ratii^ are based on a sbywnt scale in vyhich = Bcfremely Effective and "CT = Not Effecti ve. 
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Table 

Selected Faculty and Student Information 



FACULTY/STUDENT INFORMATION 


1993 RimSERS 
STATISTICS* 


1993 NATIONAL 
STATISTICS 


1993 PUBUCAAU 
STATISTICS 


1993 NORTHEAST 
PUBUCAAU 
STATISTICS 


Average number of faculty participating per 
program 


37.5 


21.0 


30.6 


31.1 


Average percentage of full professors 
participating per program 


53.9% 


50.1% 


55.5% 


49.9% 


Average percentage of faculty with research 
support per program 


28.4% 


32.8% 


39.3% 


35.0% 


Average percentage of faculty receiving an award 
or honor between 1986-1 992*' per program 


8.9% 


9.7% 


9.7% 


8.0% 


Average percentage of faculty publishing 
between 1988-1992*^ per program 


76.8% 


69.7% 


78.2% 


77.1% 


Average number of Ph.D.s produced between 
1988-1992 per program 


30.3 


29.0 


36.6 


29.2 


Average percentage of Ph.D.s awarded to 
women per program 


36.1% 


30.7% 


30.4% 


33.9% 


Average percentage of Ph.D.s awarded to 
minorities per program 


6.0% 


8.2% 


5.5% 


5.8% 


Average median time elapse (in years) in 
completing Ph.D. per program 


9.9 


9.0 


9.2 


9.5 



• Includes New Brunswick Doctoral Programs only. 

** Among Faculty from the Arts and Humanities. 

Anrxxig Faculty from all academic areas other than Arts and Humanities. 
Note:_ Nati onal and AAU s tatistics are the averages of each institution. 
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DEVELOPING A PROFILE OF RETAINED AND ATTRITED STUDENTS 



Tracy A. Hunt- White, Associate Director 
Planning and Institutional Research 
The Catholic University of America 



Statement of the Problem 

Higher education institutions are faced with declining resources and 
enrollments, and increasing numbers of academically underprepared students. 

According to Jones and Watson (1991), one of the factors critical to the survival of 
higher education institutions is full-time enrollments. They also noted that high attrition 
levels adversely affect funding, facilities planning, and long-term planning of 
curriculum (p.l). Thus, it is essential for institutions to develop policies to retain their 
full-time students, and because of limited resources, these retention policies should be 
designed based on some knowledge of the characteristics of returning and non-returning 
students. Developing a profile of returnees and non-retumees will help institutions 
create more effective and cost efficient retention policies. Also, colleges will be able to 
target the populations most likely to have high levels of attrition and determine the areas 
on which retention programs should focus. 

Purpose of the Study 

The purpose of this study is to help inform retention policies by developing a 
profile of the characteristics of returning and non-returning students. The study 
analyzed and evaluated the differences in the selected characteristics of these groups. 
Specifically, a profile of each group was created with respect to: level of satisfaction 
with college services, personal characteristics, high school background, college choice, 
academic performance, institutional support, and reasons for leaving. The following 
questions were addressed: 

1) What is the profile of returning students with respect to selected 

characteristics? 

2) What is the profile of non-returning students with respect to selected 

characteristics? 

3) What are the differences in the selected characteristics of returning 

and non-retuming students? 

Data Needs 

Cope (1978) emphasized the use of readily available and ascertainable 
characteristics of students to help identify potential dropouts. Lenning et al. (1980) 
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believed that easily-obtained data would have comparable accuracy to more 
complicated and costly measures. Data obtained easily from college admissions tests 
and questionnaires administered to students during the freshman year would also 
supply relevant information to measure persistence. Gillespie and Nobel (1992) 
added that it may be useful to collect information prior to enrollment or early in the 
first year regarding individual enrollment plans and goals in helping identify potential 
dropouts. 

Several categories of data are necessary for retention studies. They include, 
but are not limited to the following areas (Gillespie and Nobel, 1992, p.4): 

• Background information (i.e., demographics, high school GPA, etc.) 

• Initial Commitment to Institution (i.e, purpose for enrolling, choice, 

reasons for selecting college) 

• Initial and subsequent academic goal commitment 

• Student/institution academic fit (i.e., perception of relationships with 

faculty, staff) 

• Student/institution social fit (i.e., satisfaction with the environment) 

• Student/institution financial fit (i.e, type of aid, hours spent working) 

This study uses elements from most of the above categories. Data was gathered using 
easily accessible and ascertainable data from the Administrative database, freshman 
surveys, and a student opinion survey. 

Methodology 

The Office of Planning and Institutional Research created a datafile identifying 
returning and non-returning students by extracting a Fall 1 996 student enrollment file 
from the Administrative database and matching it to a Fall 1995 enrollment file 
containing full-time freshmen and sophomores. The database was also used to gather 
data elements pertaining to the personal characteristics of returning and non-retuming 
students, their high school background, their college academic performance, and their 
type of institutional support. For non-retuming students, a withdrawal student survey 
administered by the Dean of Students was used to capture reasons for leaving. 

To gather information on student satisfaction with college services, the datafile 
of returning and non-retuming students was matched to the respondents of the American 
College Testing (ACT) Student Opinion Survey which was administered in the spring 
of 1996 to full-time freshman and sophomore students. The datafile was also matched 
with the respondents of the 1994 and 1995 Cooperative Institutional Research Program 
(CIRP) Freshman Survey to provide data on the following: reasons for going to college, 
reasons for selecting this college, number of colleges applied to, and was this college 
their first choice. 



The Statistical Analysis Software (SAS) was used to generate frequencies and 
percentages pertaining to the various characteristics of returning and non-returning 
students. A chi-square test of association was performed on all categorical variables to 
test the relationship between returning status and categorical variable. The resulting 
chi-square value indicated whether the distribution of frequencies in each categorical 
variable differed significantly between each return group. 

Results 

The percentages displayed in Tables 1 through 4f serve to provide a profile of 
returning and non-retuming students and to show where these groups are significantly 
different. Table 1 provides data on the background characteristics, college 
performance, and financial support of each group. Table 2 shows the responses of 
returnees and non-retumees on selected items from the CIRP Freshman Survey. 

Table 3 indicates satisfaction with college services and programs and Table 4a 
through Table 4f show the level of satisfaction with different aspects of the college 
environment as gathered from the ACT Student Opinion Survey. Below are the tables 
and highlights from each table. Highlights of students reason for leaving are also 
provided. 

Background Characteristics. College Performance, and Financial Support - Table 1 

With respect to gender, the majority of returning and non-retuming students 
are female. Although the majority of students at CUA are white and U.S. citizens, the 
groups were different with respect to ethnicity and citizenship. Seventy-nine percent 
of returning students are white versus 68% of non-retuming students. Ninety-one 
percent of returning students are U.S. Citizens versus 87% of non-retumees. 

However, while immigrants comprise 3% of returnees, they are 8% of non-retumees. 

There was a significant difference between the religious preference of the 
groups. Eighty percent of returnees are Catholic versus 66% of non-retumees. 

There was no significant difference between returnees and non-retumees in 
how they were admitted. Eighty-four percent of returnees were admitted through non- 
conditional admissions versus 82% for non-retumees. However, there was significant 
difference between the groups regarding early decision. Twenty-four percent of 
returnees were admitted through early decision versus 11% of non-retumees. 

Returnees and non-retumees were significantly different with respect to the 
type of high school they had attended. Sixty-one percent of returnees had attended 
private schools versus 49% of non-retumees. They were also significantly different in 
high school GPA. Sixty-seven percent of returnees had a GPA of 3.0 and above 
versus 57% for non-retumees. 

Financial aid revealed a significant difference between returnees and non- 
retumees. Seventy-seven percent of returnees received aid versus 63% of non- 



returnees. Significantly fewer non-returnees received grants than returnees (45% 
versus 65%, respectively). A higher percentage of returnees versus non-retumees 
received grants rather than loans. 

TABLE 1 



Background Characteristics, College Performance, and Financial Support 





Rtn 


NRtn 




Rtn 


NRtn 


Female 


55% 


56% 


High School GPA** 












3.0 and above 


67% 


52% 


White** 


79% 


68% 


0 

1 


31% 


46% 








Below 2.0 


1% 


1% 


Citizenship* 












US Citizen 


91% 


87% 


College GPA** 






Immigrant 


3% 


8% 


3.0 and above 


51% 


37% 


NRA 


5% 


5% 


0 

1 


45% 


37% 








Below 2.0 


4% 


27% 


Catholic** 


. 80% 


66% 














Financial Aid 






Early Decision* 


24% 


11% 


Receiving Aid** 


77% 


63% 








Grants** 


70% 


53% 


Regularly Admitted 


84% 


83% 


Loans** 


65% 


51% 








Institutional Aid** 


65% 


45% 


Private H.S,** 


61% 


48% 


Government Aid** 


66% 


52% 



*Significant at p<.05; **Significant at p<.01 Rtn=Retumed; NRtn=Attrited 



Freshman Survey: College Choice. Reasons for Attending College and CUA - Table 2 

Seventy-four percent of returnees and 72% of non-retumees selected CUA as 
their first choice. Only 2% each of returnees and non-retumees indicated that CUA 
was less than third choice. Slightly more returnees than non-retumees applied to four 
or more other colleges (61% vs. 57%, respectively). 

Almost all of the returnees and non-retumees chose to "learn more about 
things" and to "gain general education" as the top reasons for attending college. More 
returnees than non-retumees indicated to "make more money" as a top reason for 
attending college (93% vs. 88%, respectively). While 97% of non-retumees selected 
to "get a better job" as a top reason (88% of returnees). 

Also note that 69% of non-retumees selected "wanted to get away from home" 
as a reason for going to college versus 63% of returnees. 

Both returning and non-returning groups gave the following as the top 
important or very important reasons for attending CUA: “Good academic 
reputation,” “Graduates get good jobs,” and “Size of institution.” 
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Returnees and non-retumees were significantly different with respect to how 
important "offered financial aid" was as a reason for attending CUA: 77% of 
returnees and 59% of non-retumees. They were also Significantly different with 
respect to the importance of "college representative recmitment" as a reason for 
attending CUA: 25% of returnees and 12% of non-retumees. 

TABLE 2 



College Choice, Applications, Reasons for Attending College and CUA 





Rtn 


NRtn 




Rtn 


NRtn 


Choice of College 






Reasons for Choosing CUA 






Less than 3rd choice 


2% 


2% (% indicated ImptA^ ery Impt) 






3rd choice 


3% 


4% 


Good academic reputation 


98% 


100% 


2nd choice 


21% 


23% 


Graduates get good jobs 


92% 


90% 


1 St choice 


74% 


72% 


Size of institution 


91% 


88% 








Graduates go to top grd schs 


84% 


81% 


Nbr of Other Coll Applns 






Offered financial assistance** 


77% 


59% 


3 or less 


39% 


43% 


Social reputation 


75% 


76% 


4 or more 


61% 


57% 


Religious affiliation 


69% 


59% 








Special educational programs 


60% 


49% 


Reasons for Attending Coll 






Advice of guidance counselor 


36% 


40% 


(% indicated ImptA^ ery Impt) 






Relatives desired it 


36% 


37% 


Learn more about things 


99% 


100% 


Advice of teacher 


36% 


35% 


Gain general education 


97% 


100% 


Friend suggested it 


29% 


25% 


Make more money 


93% 


88% 


Low tuition 


27% 


23% 


Become more cultured 


93% 


98% 


College rep recmitment* 


25% 


12% 


Get a better job 


88% 


97% 


Wanted to live near home 


24% 


23% 


Improve reading/study skills 


87% 


88% 


Athletic recmitment 


20% 


16% 


Parents desired it 


73% 


72% 


Advice of priv coll counselor 


10% 


14% 


Get away from home 


63% 


69% 


Not accepted anywhere else 


5% 


2% 


Mentor encouraged me 


47% 


36% 









*Significant at p<.05; **Significant at p<.01 Rtn=Retumed; NRtn=Attrited 



Satisfaction with College Services — Table 3 

Returnees and non-retumees were not significantly different with respect to 
their level of satisfaction with college services. More returnees than non-retumees 
were satisfied with recreational/intramural program (87% vs. 73%, respectively) and 
student employment services (81% vs. 70%, respectively). Also, both groups had a 
low level of satisfaction with food services (20% of returnees and 1 9% of non- 
retumees). 

Returning students had a higher level of satisfaction with resident hall services 
than non-retumees; 57% of returnees versus 35% of non-retumees. The same 
relationship was tme for college sponsored social activities, 58% of returnees and 
42% of non-retumees were satisfied. 
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TABLES 

Satisfaction with College Services or Programs 



Rtn NRtn 


Recreational/Intramural programs 


87% 


73% 


Student employment services 


81% 


70% 


Academic advising services 


69% 


71% 


College orientation program 


67% 


65% 


Computer services 


64% 


82% 


College-sponsored social activities 


58% 


42% 


Residence hall services and programs 


57% 


35% 


Financial aid services 


55% 


53% 


Library facilities and services 


47% 


43% 


Food services 


20% 


19% 



Rtn=Retumed; NRtn=Attrited 



Satisfaction with College Environment — Tables 4a - 4f 
Academic — Table 4a 

Non-returning satisfaction had the lowest level of satisfaction with the 
following areas: Variety of courses offered by the college (33% of non-returnees 
and 63% of returnees). 

TABLE 4a 

Satisfaction with College Environment - Academic 



Rtn NRtn 



Testing/grading system 


67% 


53% 


Course content in major field* 


76% 


56% 


Instmction in major field 


76% 


63% 


Availability of instmctor outside of class 


73% 


59% 


Attitude of faculty toward students 


76% 


67% 


Variety of course offered by college** 


63% 


33% 


Class size relative to type of course 


84% 


70% 


Flexibility to design program of study 


54% 


36% 


Availability of advisor 


66% 


63% 


Value of the information provided by advisor 


63% 


53% 


Preparation receiving for future occupation 


61% 


66% 



* Significant at p<.05; ** Significant at p<.01 Rtn=Retumed; NRtn=Attrited 

Returnees and non-returnees were significantly different with their level of 
satisfaction with course content in their major field. Seventy-six percent of returnees 
were satisfied versus 56% of non-returnees. 

The non-retumees and returnees were significantly different with respect to 
their level of satisfaction with the flexibility to design program of study (36% of non- 
retumees and 54% of returnees). 

Admissions — Table 4b 
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Both returnees and non-retumee groups had the highest level of satisfaction 
with the general admissions process (66% and 50%, respectively). Both groups also 
had the lowest level of satisfaction with the availability of financial information prior 
to enrolling (51% of returnees and 38% of non-retumees). 

Overall, non-retumees had low levels of satisfaction with all aspects of the 
admission process. The groups were significantly different in their satisfaction with 
the college catalog/admissions publications (64% of returnees and 43% of non- 
retumees). 



TABLE 4b 



Satisfaction with College Environment - Admissions 



Rtn NRtn 


General admissions procedures 


66% 


50% 


Availability of financial aid info prior to enrolling 


51% 


38% 


Accuracy of college info received before enrolling 


63% 


47% 


College Catalog/admissions pubs* 


64% 


43% 



♦Significant at p<.05; **Significant at p<.01 Rtn=Retumed; NRtn=Attrited 



Rules and Regulations — Table 4c 

Very few of the students were satisfied with the mles and regulations on 
campus. The area with the highest satisfaction for returnees was academic probation 
and suspension policies (38% for returnees vs. 22% for non-retumees). The area with 
the highest satisfaction for non-retumees was rules governing student conduct (37% 
of non-retumees and 32% of returnees). 

TABLE 4c 

S atisfaction with College Environment - Rules & Regulation s 

Rtn NRtn 



Student voice in college policies 
Rules governing student conduct 
Residence hall mles and regulation 
Academic probation/suspension policies 
Purposes for which student activity fees used 

Personal security/safety on campus 

Rtn=Retumed; NRtn=Attrited 



32% 21% 
32% 37% 
30% 27% 
38% 22% 
29% 26% 
27% 30% 



Personal security/safety on campus was the area with the lowest area of 
satisfaction for returnees (27% for returnees and 30% for non-retumees). Student 
voice in college policies was the area with the lowest level of satisfaction for non- 
retumees (21% of non-retumees vs. 32% of returnees). 
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Facilities — Table 4d 

Returnees had the highest level of satisfaction with athletic facilities (68% of 
returnees vs. 59% of non-returnees). Non-returnees had the highest level of 
satisfaction with the college bookstore (67% of non-returnees vs. 56% of returnees). 

TABLE 4d 

Satisfaction with College Environment - Facilities 



Rtn NRtn 



Classroom facilities 


67% 


57% 


Laboratory facilities 


54% 


39% 


Athletic facilities 


68% 


59% 


Study area 


62% 


52% 


Student union 


32% 


20% 


Campus bookstore 


56% 


67% 


Availability of student housing 


54% 


45% 


General condition of buildings/grounds 


48% 


33% 



Rtn=Retumed; NRtn=Attrited 



Both groups were least satisfied with the student union (32% of returnees vs. 
20% of returnees) and the general condition of buildings and grounds (48% returnees 
and 33% non-retumees). Also, only 39% of non-retumees were satisfied with 
laboratory facilities versus 54% of returnees. 

Registration — Table 4e 

Returnees and non-returnees had the highest satisfaction with the academic 
calendar (65% and 60%, respectively). While 59% of returnees were satisfied with 
“general registration procedures” versus 40% of non-retumees). 

TABLE 4e 



Satisfaction with College Environment - Registration 



Rtn NRtn 


General registration procedures 


59% 


40% 


Availability of courses when needed 


49% 


47% 


Academic calendar for this college 


65% 


60% 


Billing and fee payment procedures 


37% 


23% 



Rtn=Retumed; NRtn=Attrited 

Both returnee and non-retumee groups had the lowest satisfaction with billing 
and fee payment procedures (37% and 23%, respectively). 

General Aspects of the College Environment — Table 4f 

There was a significant difference between non-retumees and returnees with 
the level of satisfaction with racial harmony at CUA (32% of non-retumee vs. 55% of 
returnees). There was also a significant difference between the groups with respect to 
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level of satisfaction with CUA in general (38% of non-retumees vs. 70% of 
returnees). 



TABLE 4f 

Satisfaction with College Environment - General 



Rtn NRtn 



Concern for you as an individual 


46% 


34% 


Attitude of staff toward students 


49% 


48% 


Racial harmony at this college* 


55% 


32% 


Opportunities for student employment* 


61% 


41% 


Oppties for involvement in campus activities 


62% 


45% 


Student government 


42% 


30% 


Religious activities and programs 


59% 


50% 


Campus media 


53% 


48% 


This college in general** 


70% 


38% 



* Significant at p<.05; ** Significant at p<.01 Rtn=Retumed; NRtn=Attrited 

Returnees and non-retumees were almost equally satisfied with the attitude of 
staff toward students (49% and 48%, respectively). Both groups had the lowest level 
of satisfaction with student government (42% of returnees vs. 30% of non-retumees). 

Non-returning students were the least satisfied with opportunities for student 
employment (41% of non-retumees vs. 61% of returnees) and opportunities for 
personal involvement in campus activities (45% of non-retumees vs. 62% of 
returnees). 

Reasons for Leaving 

To capture data on students reason for leaving, students who withdrew were 
asked to fill out a survey as part of the withdrawal process. The results from the 
survey are provided with the knowledge that respondents may not be a representative 
sample of those who dropped out and also that they may not provide the "real" reason 
for leaving. 

Most of the respondents indicated that they were transferring to another 
school. Some of the major issues with which respondents were dissatisfied included 
uncomfortableness with the social atmosphere; lack of social activities on-campus; 
and lack of diversity. Several respondents cited that they did not feel as if they 
belonged. Other frequently mentioned issues were drinking by friends and 
roommates and concern for personal safety on and off campus. 

Summary and Conclusion 

The findings show noticeable differences between returnees and non- 
retumees. Non-retumees are significantly different from returnees in ethnicity and 
citizenship and were more likely to have attended public high school. In addition. 
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fewer non-retumees received financial aid than returnees and were less likely to be 
awarded grants. 

More non-retumees than returnees were dissatisfied with college sponsored 
social activities, course content in major field, and the variety of courses offered. 

They were also less likely to be satisfied with the flexibility to design their program of 
study. 



Non-retumees were less satisfied than returnees with all aspects of the 
admissions process. They were particularly dissatisfied with the availability of 
financial aid information prior to enrolling. Also, they were more likely to be 
dissatisfied with CUA in general, the registration process, racial harmony at CUA, 
and opportunities for student employment. 

In conclusion, the basic attempt of this study was to develop a profile of 
returning and non-returning students using widely available resources to which most 
institutional research offices have access and to show some of the variables that are 
commonly used in retention studies and could be used in developing models to 
predict retention. 

Further Research 

Further research to help develop effective retention policies could include 
administering a survey to attrited students and obtaining an identifier to allow tracking. 
A longitudinal profile of returning and non-returning students should be developed and 
maintained. Institutions could evaluate the effectiveness of retention policies before and 
after revising programs based on knowledge of the characteristics of its returning 
students. Finally, regression models could be developed to predict a student's likelihood 
of returning. 
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Graduation Rates at the University of New Hampshire 
An Historical Perspective - Mortality in Retrospect 



John Kraus and Antonietta Taylor 
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University of New Hampshire 



Introduction 

Graduation rates are now gauges of perceived institutional goodness. This data 
coin has two sides. Expressed in a positive way, results are graduation or 
completion rates. These are the students that make it through an institution to 
degree completion (sometimes said to be "persisters"). The opposite side of this 
are those that don't finish. They make up the attrition or withdrawal rate (these are 
also called "leavers" or "dropouts"). As aspects of institutional effectiveness these 
rates are used for peer comparison and in various rankings such as in U.S. News 
and World Report. The Student Right-to-Know and Campus Security Act at the 
federal level and the NCAA Report on Graduation Rates both attest to the 
escalation of concern about how many students finish their programs of study. 
Media hype has inflamed perceptions. Legislators, governing boards, central 
offices and others have joined the fray. 

What about graduation rates in the past, sixty or seventy years ago? Wouldn't 
this be relevant to provide a reference point for today? It could be especially 
important as a backdrop in conversations with those who say that we aren't doing 
as good a job as "previously." Curious, we determined to discover what 
graduation rates were at the University of New Hampshire in the 1920s and 
1930s. We also would search the literature for comparable data. 

Ours is a very straightforward study - select cohorts and track students in them 
by name through six years at UNH, looking for individual graduation, then 
compare the results with current rates. We used three cohort years - 1924, 1929 
and 1931. The comparison indicates that at UNH we are doing a better job of 
graduating students now than in the 1920s or 30s. We rediscovered the seminal 
investigation for this period and find that our historical rates compare favorably 
with what it indicates. Hopefully our endeavor will provoke some interest in 
historical aspects of the graduation rate issue. 

Background 

A review Of the literature suggests that retention and dropout studies came into 
the fore in the 1960s and rapidly became a major issue in higher education. 
Vincent Tinto indicates that for many this interest sprang from "... a growing 
realization that in the absence of growing pools of applicants the financial health 
of their institution is very much affected by their ability to retain a greater portion 
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of entering students until degree completion. For a number of institutions, 
particularly those smaller colleges which admit virtually anyone who applies, the 
issue of dropout and retention is tantamount to that of institutional survival." 
(Tinto, 1982) 

In the 1970s retention research burgeoned. The literature ballooned. Retention 
task forces and attendance at special attrition or retention conferences or seminars 
became vogue. Efforts now include comprehensive institutional studies such as 
the Consortium for Student Retention Data Exchange (CSRDE) established in 
1994 at the University of Oklahoma, with over 200 members, and the National 
Graduation Rate Study at the University of Arizona with over 50 land grant or 
public research universities participating. 

What about the period of interest for this study? Were there discussions about 
graduation rates or retention in the 1920s or 30s? College (Gavit, 1925) is 
certainly one typical discourse. A review of the index finds reference to "gangs" 
and "gifted students" but nothing about graduation rates. There is mention of 
"alumni" and "athletics" but not of attrition. "Drifting" and "drinking" are cited 
but dropouts is not indicated. Where are these issues covered? The answer is 
under "mortality," a rather ironic application of the term. This word certainly 
offers a sense of finality not even conveyed in today's use of "dropout." 

One of the abiding causes of the assumption of incompatibility of interest 
between the student and the teacher-caste is the atmosphere surrounding 
the subject of examinations and the associated subject of institutional 
"mortality." Personally I am unable to understand the complacency with 
which school and college officers and teachers view these things. I know a 
school where certain influential members of the faculty all but boast of 

grim satisfaction in what they call "the annual blood-letting." 

• • • 

I don't know that anybody ever has made a study of the economic waste of 
the "turnover" and spoilage of material in the business of education, 

through the permanent moral maiming of the victims of this sort of thing. 

• • • 

...Perish the thought that every possible effort ought to be made to 
conserve and develop what the student has\ to find out and take advantage 
of what he has acquired. (Gavit, T 925) 

This passage vividly captures the spirit of what higher education is trying to do 
today in retention efforts - to avoid the economic waste of student "turnover." 

Gone is the apparent luxury (or stupidity) of purging classes through academic or 
other kinds of hazing. Every student now counts. 

Reviews uncovered a single pre World War II study of graduation rates. It is a 
real gem and, of course, features the key word - mortality. One of a series of U.S. 
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Office of Education studies undertaken during 1936-37 with financing under the 
Emergency Relief Appropriations Act of 1935, College Student Mortality 
(McNeely, 1938) is a 112 page bulletin outlining the results of a survey of twenty- 
five universities. Four year graduation data was collected on first-time full-time 
students in the freshman class registered for a bachelors degree at the opening of 
the 1931-32 academic year. Students not graduating were also tracked for one 
additional year beyond the four. (By wonderful coincidence, our third cohort is 
also the entering class of 193 1 .) The study was very comprehensive, with data on 
15,535 individual students. It included differences by sex, by years attending, by 
institutional school or college (discipline), by age, location, lodging, academic 
achievement, credit hours, and by cause (financial, etc). Participating were 14 
publicly controlled and 1 1 privately controlled institutions. The results of the 
study are much too detailed for treatment within the limits of this paper but 
summary data will be displayed with that for UNH as we discuss our results. 
Overall the percentage of students "obtaining degree during or at the end of 4-year 
period" was indicated to be 28.3 percent for publicly controlled institutions and 
36.4 for private schools. By institution the range was from the high of 57.8 
percent to 13.8 percent for the lowest school. 

McNeely also confirmed our suspicions that little work on "mortality" had 
preceded that effort. "Most of the previous studies on this subject have been 
narrow in scope, being confined to a single institution rather than a group of 
institutions. Thus little is known as to the differences in student mortality among 
institutions or the extent to which students leave college generally throughout the 
United States" (McNeely, 1938). An example institutional study was cited as one 
by Ohio State University in 1929. A limited survey of land grant colleges and 
universities was noted as having been conducted by the Office of Education in 
1927-29. 

We also found that Tinto (1982) developed and graphed "... data on the overall 
rates of first four-year degree completion of students in higher education over the 
period 1880-1980. Degree completion rates are calculated by the ratio of the 
number of BA's or first professional degrees given in any year to the number of 
first-time degree enrollments four-years earlier." Tinto does not specify his source 
for the numbers but it must have been summary data from government documents. 
This is an aggregate method of estimating, omitting by name tracking. His result 
is that "... rates of completion (dropout) have, with the exception of one period, 
remained surprisingly constant over one hundred years at about fifty-five (forty- 
five) percent." He contends that based on this historical pattern which is also 
what the rate was in the early 1980s "... we should be much more skeptical of 
optimistic projections of our ability to significantly reduce dropout in higher 
education at the national level." In other words, there is no room for 
improvement. It seems to us, as a result of analysis of UNH data and of reviewing 
the McNeely study (with an overall four year graduation rate of 3 1 .6 percent) that 
there has been more room for improvement than Tinto's estimation suggests. 



Methodology 

As our cohorts, we used the entering freshman classes of 1924, 1929 and 1931. 
We started with 1929. The real trick was to find a reliable source for this antique 
information. The principal document for our data was a vintage item, the Bulletin 
of the University of New Hampshire, Catalog. We were fortunate. The catalog in 
the early years included a by name list of all students by class, with college/major, 
and town and state of residence. Additionally the catalog listed all students who 
received a degree during the previous year, with separate listings for baccalaureate 
and associate level students. Our effort only considered baccalaureate level 
students. 

Initially we thought it would be ideal to select a cohort for tracking from each 
decade in the century. The reality of historical turbulence and data access dictated 
otherwise. Looking closely at the 1900-09 decade it was clear that the numbers 
were sufficiently small to make any association with current enrollment a real 
stretch (Catalogs in this period were, however, wonderful resources containing 
both complete registers and alphabetical listings of all graduates.). The next 
decade was disrupted by World War I. The twenties and the thirties were 
consistent but the Depression might have some influence on graduation rates. In 
the late twenties and early thirties the economy was certainly a concern. The 
university history states that "... the number of tuition scholarships was 
increased to 225 in 1928-29 and to 250 the following year. By 1933-34, the 
general economic situation was so bad that the amount available for tuition loans 
was increased . . . ." (Sackett, 1974). 

Then we discovered that, with the 1938 issue, the catalog discontinued listing 
the names of students. By serendipity, perhaps that was a good end point since 
World War II would scramble enrollment patterns for the 1940s. And from 1938 
forward, we found no ready source of either freshman rosters or graduation 
names. Not able to undertake major archival mining, we settled on three cohorts 
from the 1920s and 1930s. Our tracking efforts, however, would, in the end, 
cover 14 years in those decades. 

For an independent checking mechanism of catalog information, we referred to 
our UNH Factbook which includes historical data for the University back to 1 868. 
We knew that those numbers were originally drawn from Registrar's Office 
summaries. Reviewing counts by class we confirmed that the catalog was a valid 
source. Counts were always within a handful in both sources (and most often 
there was only a difference of one or two in each year). Also, with handwritten 
changes and updates made directly to these earlier undergraduate catalogs, it 
appeared that these were official, annotated, working documents. 

All freshman class names (last name, first and middle initial) for each year 
were entered into an ASCII file, along with the degree or college indicated, and 
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gender. We assigned gender and, as you can imagine, in some cases it was 
difficult to determine whether a name was male or female; however, the middle 
name usually cleared up any confusion. Once all the names were typed in, they 
were closely hand checked against the catalog for errors or omissions. The ASCII 
files were then loaded into System 1032 data bases to establish each freshman 
cohort. 

The same basic procedure was followed for the graduates, at least initially. All 
names for the graduating classes four years out from the corresponding freshman 
class were entered into ASCII files. We included last and first name, and middle 
initial, the degree awarded and the year of graduation. Those separate files were 
also loaded into graduation data bases. 

Extensive manual checking and editing was required. This was time- 
consuming careful work. Sometimes a first, middle or last name for a freshman 
record was slightly different from what was indicated for that person in the 
graduation data. Since mapping from the freshman data base to the graduation 
data base would only identify exact matches in both data sets, we reviewed all 
non-matching entries individually to make sure we had indeed captured everyone. 
We found many instances where spelling of names in the freshman data base was 
different than in the graduation data. Such anomalies were carefully reviewed and 
records were altered if needed for consistency. If we determined that a difference 
in last name was through marriage or other change in status, we amended the 
records to be consistent in both data sets. In several instances, "Jr." was denoted 
in one data set but not the other. In these cases, we went back to the catalogs and 
if we determined that the person was one and the same, we altered the records 
(e.g. freshman record: Isherwood, Jr., William Lea from Berlin in the College of 
Technology; graduation record; Isherwood, William Lea from Berlin in the 
College of Technology— hard to mistake this!). Fortunately we didn't have to rely 
on just the names. We could check town and state and also major to help verify 
entries. Once we completed these cross-checking efforts, we were confident that 
we'd identified all freshmen who graduated in four years. 

Our next step was to include three, five and six year graduation rates. To do 
this we started with the list of freshmen not graduating in four years and manually 
checked the individual names against the graduation lists for those additional 
years in the catalog. If they were identified in the catalog, we added the entries to 
the graduation database. We did find instances where first and middle names 
were reversed between the freshman and graduation lists. Again, by referring to 
the catalog and checking the entry by town/state of residence and major/college 
we were able to determine if the persons were one and the same, and if so, records 
were changed for consistency. 

We now had three, four, five and six year graduation data (curiously, three to 
five percent of the freshmen cohorts graduated with a baccalaureate within three 
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years). By interactively mapping back and forth, we calculated rates by gender as 
well as for the total. The tables below display graduation rates for the three 
historical class cohorts investigated and for three recent 1980s cohorts for 
comparison. 



Fall 

of Year 


# 

Freshmen 


# 

Grad in 
4 years 


% 

Grad in 
4years 


# 

Grad in • 
5 years 


% 

Grad in 
5 years 


# 

Grad in 
6 years 


% 

Grad in 
6 years 


Historical Graduation Rates 


1924-25 
















Men 


370 


142 


38.4% 


172 


46.5% 


181 


48.9% 


Women 


136 


90 


66.2% 


94 


69.1% 


94 


69.1% 


Total 


506 


232 


45.9% 


266 


52.6% 


275 


54.4% 


1929-30 
















Men 


329 


151 


45.9% 


181 


55.0% 


189 


57.5% 


Women 


129 


76 


58.9% 


83 


64.3% 


83 


64.3% 


Total 


458 


227 


49.6% 


264 


57.6% 


272 


59.4% 


1931-32 
















Men 


396 


167 


42.2% 


185 


46.7% 


188 


47.5% 


Women 


119 


68 


57.1% 


69 


58.0% 


69 


58.0% 


Total 


515 


235 


45.6% 


254 


49.3% 


257 


49.9% 



Recent Graduation Rates 


1987-88 
















Men 


945 


451 


47.7% 


616 


65.2% 


655 


69.3% 


Women 


1234 


767 


62.2% 


915 


74.2% 


939 


76.1% 


Total 

1988-89 


2179 


1218 


55.9% 


1531 


70.3% 


1594 


73.2% 


Men 


928 


440 


47.4% 


618 


66.6% 


647 


69.7% 


Women 


1188 


685 


57.7% 


869 


73.2% 


898 


75.6% 


Total 

1989-90 


2116 


1125 


53.2% 


1487 


70.3% 


1545 


73.0% 


Men 


965 


481 


49.8% 


663 


68.7% 


694 


71.9% 


Women 


1270 


777 


61.2% 


943 


74.3% 


960 


75.6% 


Total 


2235 


1258 


56.3% 


1606 


71.9% 


1654 


74.0% 



Discussion 

Three things stand out in looking at the 1920s - 30s graduation data. First, 
averaging the three 1980s cohorts, the UNH six year graduation rate today is 19 
percent higher than an average of the three historical cohorts studied (73.4 percent 
to 54.4 percent). The four year rate is also higher for the three current cohorts 
(55.1 percent to 46.9 percent). Certainly on the face of this data, we are doing a 
better job now than then. Second, women had higher graduation rates than men. 
This should be no surprise. The fact that women are more likely than men to 
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finish a bachelor's program has been well documented in national studies by Astin 
and others. Our data and that of McNeely confirm that historically women were 
more likely to complete a bachelor's degree. Third, the participation rate for 
women freshmen at UNH is now double what it was for the cohorts reviewed. At 
best, for 1929, women represented 28.2 percent of the bachelor's freshmen. In the 
1980s cohorts it runs a consistent 56 to 57 percent of the freshman total. 

The movement of women into higher education escalated in the twenties. A 
quick review of University of New Hampshire freshmen classes for percentage of 
women shows: 

Fall 1900 2.4% 

Fall 1910 6.6% 

Fall 1920 25.0% 

In terms of actual counts, UNH went from 1 woman in the 1900 incoming 
freshmen group to 82 freshmen women in 1920. The Revolt of Modern Youth 
(1925), highlighted the new attitude, "... the Flapper who makes her ovm living, 
votes, holds her own in competition with men, refuses to let the corset makers put 
stays on her, and snaps her fingers at 'styles' dictated by the makers of clothes, is 
capable of doing things her mother couldn't come within sight of." 

Our review touched every year from 1924 to 1937 as we tracked the three 
cohorts -1924, 1929 and 1931 - through six years. Without getting into detail with 
socioeconomic or historical aspects, some educated guesses may explain the 
variation in graduation rates for these cohorts as opposed to the relative 
consistency for our current cohorts. Simply said, the stock market crashed in 
October 1929 and, by 1931, the country was heading into the Depression. 

Compare that, for the 1929 cohort, 45 additional students (9.8 percent) graduated 
after four and before six years but only 22 additional students (4.3 percent) 
completed their degrees in the two following years for the 1931 cohort. This 
likely reflects economic realities as students failed to return for lack of money or 
to help in a family business or on the farm. A supplementary study by the 
University of Louisville found an eight percent increase in mortality, tracking the 
difference in the 1930, 1931 and 1932 cohorts (McNeely, 1938). 

The decline in the proportion of female students from almost 27 percent of the 
freshman cohort of 1924 to 23 percent in 1931 is probably another effect of the 
Depression. Families with less money were less likely to send daughters to the 
university. Also the proportion of females of the total graduating within six 
declined at an ever steeper pace, from 34.2 percent for 1924 to 26.8 for 1931. 
Women were apparently less able to finish a degree program. It also could be 
hypothesized that with economic difficulties, opportunities for women 
diminished, more got married (the old saw of "getting an MRS.") and dropped 
out. Whatever the reasons, the trend is most vivid in comparing the 66.18 percent 
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of women graduating within four years for the 1924 cohort (finishing in 1928) 
with 57.14 percent finishing in four years for the 1931 cohort. 

The four year graduation percentage for men, however, actually improved by 
four to seven percent . Perhaps with the excesses of the Roaring Twenties a thing 
of the past, those men attending were now more motivated and earnest in their 
efforts. Freshman enrollment did drop 9.5 percent, from 506 in 1924 to 458 in 
1929. The drop for men was 1 1 percent and for women it was 5.1 percent. 

The fortunate circumstance of McNeely using the 1931 freshman cohort 
enables us to compare UNH results for 1931 with that study. The table following 
provides percentages of students graduating within four years for selected 
institutions from McNeely and for UNH. 



1931 Freshman Cohort 

Percent Graduating in Four Years 

Institution 

Iowa State College 

Massachusetts State College 

Pennsylvania State College 

University of Wyoming 

University of Illinois 

Rutgers University 

Syracuse University 

Washington University 



Men 


Women 


Total 


19.9% 


25.5% 


21.4% 


57.8% 


49.4% 


55.3% 


53.4% 


69.3% 


56.0% 


17.3% 


13.7% 


15.9% 


25.9% 


30.2% 


27.1% 


57.5% 


58.5% 


57.8% 


50.4% 


55.6% 


52.5% 


26.5% 


39.7% 


31.6% 



University of New Hampshire 42.2% 57.1% 45.6% 



The institutions selected are toward the high end of the McNeely study and/or are 
most like UNH in type or geographical location. The results show UNH data to 
be compatible with that study and demonstrate variation by institutional type and 
location. 

Finally, it seems clear that in the 1920s and 1930s if one didn't graduate in the 
traditional four year cycle one was much less likely to ever finish. Only four to 
ten percent of the three historical UNH cohorts studied graduated after four years 
but within six years. For the three 1980s cohorts it ranged from 17 to almost 20 
percent of the cohort finishing after four but within six. Certainly in the twenties 
and thirties a high school diploma was considered a mark of success. Even 
without graduation, some college was a further enhancement. McNeely observes, 
"Of the 25 universities [studied], there were 7 which did not report any students 
continued beyond the 4-year period.... It is evident, therefore, that the universities 
maintained different policies with respect to encouraging students to continue in 
the institutions after failing to graduate within the required time. In slightly more 
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than one-fourth of the institutions, a definite policy apparently prevailed against 
encouraging students to remain beyond the regular 4-year period. Most of them 
were privately controlled universities." (McNeely, 1925) 

Conclusion 

The singular result of this analysis is that the University of New Hampshire 
produces about 20 percent more graduates in six years for every entering freshman 
class today than it did in the 1920s and 30s. It also seems, in reviewing the data 
from the McNeely study, that higher education in general has reduced attrition and 
improved graduation rates. 

McNeely indicates a 31 .6 percent average four year graduation rate for the Fall 
1931 freshman cohort of twenty-five universities. Recent four year rates are 
variously reported, depending on institutional grouping, as 34.4 percent for public 
universities and 69.2 for private universities (Astin, 1996), 27 percent for public 
institutions and 41 percent for private institutions (CSRDE, 1996) and 29.6 
percent for land grant and research universities (Kroc, 1995). Astin's overall rate 
is 39.9 percent for the 1985 cohort. Citing the rate of 46.7 percent for the 1966 
freshman cohort, he states that "... [comparing these] results reinforce[s] the 
popular conception that degree completion rates in American higher education 
have been declining." Nevertheless, current rates are largely better than those of 
sixty or seventy years ago. 

We like to believe that higher education has made solid progress in conserving 
its student resources - in Gavit's words, "... the economic waste of the 'turnover' 
and spoilage of material in the business of education." We can do more, but we 
should take heart that we are making headway. Attrition is an issue of serious 
concern. We have improved "mortality." Perhaps additional studies by individual 
institutions and in sequences of decades can further chart our gains and bring 
greater understanding of changes in graduation rates. 
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A SECTOR-WIDE SURVEY OF FACULTY IN PRIVATE 
HIGHER EDUCATION: ONE STATE'S THOUGHTS ON THE 
NATURE OF FACULTY WORK AND REWARD SYSTEMS 



Michael McGuire and Jason Casey 

Pennsylvania Independent College and University Research Center (PICURC) 

Introduction 

Higher education and its faculty have been closely scrutinized during the past fifteen 
years. Areas of concern have included the escalating price of a college education; the 
uncertain skill levels of college graduates and disappointing short-term employment 
prospects for some; the perception that faculty devote too much of their time to 
research and graduate programs and too little to teaching undergraduates; mixed 
messages about the adequacy and allocation of instructional resources; and the belief 
that faculty are being asked to do too much, spreading their energies across too many 
activities to the detriment of each one. As Wergin (1994) noted: 

“College faculty in particular have borne the brunt of the criticism. We 
are sheltered, spoiled folk with cushy jobs, the perception goes; we 
probably don’t work very hard, and if we do, we’re more interested in 
narrow, inaccessible scholarship than in work that addresses society’s 
problems, and we’re more interested in specialized graduate education that 
fits those narrow specialties than we are in teaching undergraduates.” (p. 

1 ) 

This criticism has not gone unnoticed by policy-makers. Russell (1992) reports on a 
survey of state higher education executive officers that revealed significant activity in 
many states to study or implement policies and standards related to faculty workload, 
tenure, and evaluation. Cahir and colleagues (Permsylvania State University, 1994) 
referred to mounting evidence of closer state supervision of higher education in 23 
states, and in particular, cited legislative involvement in faculty workload matters in 
five states. In Pennsylvania, a series of legislative hearings in 1995 led by 
Representative John Lawless (Marshall, 1995; Moran, 1995) raised serious questions 
about institutional policies and faculty productivity in the areas of teaching loads, 
tuition benefits, and sabbatical leaves. 

Understanding the Issues: Recent Contributions 

Public criticism has spurred significant study of these perceived problems within the 
academy. This study has taken several forms: essays on conceptualization and 
methodology, research results, and on-campus roundtable discussions. 

The State Policy and College Learning (SPCL) project was initiated by the Education 
Commission of the States, with primary funding support from The Pew Charitable 
Trusts. The 1992 report, “An Agenda for Reshaping Faculty Productivity” 
(Heydinger and Simsek, 1992), emphasized the need to change the current research 
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paradigm that has dominated the American university landscape in the twentieth 
century, particularly since the end of World War II. This call to action included an 
insightful comparison of the professoriate to other professions — physicians, lawyers, 
and ministers — that further reinforced the inevitability of change within the higher 
education system that will parallel changes that other professions are already 
experiencing. Secondary comparisons of the work habits and reward systems of 
college faculty versus other professions — salespeople, performing artists, professional 
athletes, police officers, and commercial airline pilots — further supported the notion 
that higher education is not exempt from the cultural and market forces driving 
adaptation in other employment sectors; 

“Many industries are delivering significantly more value without 
commensurate price increases (i.e., electronics, package delivery, fast 
food). Other industries are increasing their prices but are also delivering 
more value (i.e., restaurants, specialty tours). All of this is being done 
with a pace and quality that reflect the changing lifestyles around the 
globe. People are becoming accustomed to high-quality, quick-response 
service in which their needs are met.” (p. 1 5) 

The authors offer a comprehensive agenda for reshaping faculty productivity that uses 
incentive schedules to retain flexibility in work definition, reward effectiveness, and 
require individual accountability. 

Another recent volume that contributes both diagnostic and prescriptive information 
to the debate about faculty productivity is the 1994 publication, “Analyzing Faculty 
Workload,” edited by Jon Wergin. Of special interest are discussions of 
methodologies for measuring faculty time allocation (Jordan, 1994) and an in-depth 
case study of faculty productivity measurement at one large research-oriented 
university (Glazer and Henry, 1 994). What becomes clear from reading these papers 
is that higher education, if so motivated, already has the tools to move the 
productivity debate to empirical rather than merely philosophical ground. The data 
that are offered suggest that negative public perceptions are not generally valid: 
faculty do work long hours, and devote significant time and energy to teaching. In 
response to the concern that many workload studies rely on self-reported data with 
little or no objective validation of faculty claims, Jordan notes: 

“... critics of self-report studies suggest that the use of self-reported data 
leads to inflated workload results. Supporters of self-reported data believe 
that consistency of responses over long periods of time lends validity to 
the typical findings that faculty work fi^-two to fifty-seven hours per 
week.” (p. 1 8) 

It is important to remember that criticism of the faculty activity and reward system 
does not come only from external sources. Several studies have documented faculty 
dissatisfaction with their compensation, the imbalance between teaching and research 
in the reward system, and the number and variety of stressors in their jobs (Dey, 
Ramirez, Korn, and Astin, 1993; Gray, Froh, and Diamond, 1992; Gray, Diamond, 
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and Adam, 1996; Pennsylvania State University, 1994; National Center for Education 
Statistics, 1994). It would be inaccurate to assume that faculty are content with the 
status quo and that only those outside of the academy are clamoring for change. 

Nor are many institutions mired in the status quo. In their volume Recognizing 
Faculty Work: Reward Systems for the Year 2000, Diamond and Adam (1993) cite 
both examples of institutions that have successfully reformed their reward systems 
and models for understanding and implementing change. There is no shortage of 
methodologies for improvement; the problems on many campuses seem to be political 
and cultural. 

Recent National Research 

The topics of faculty productivity and reward systems cover enormous research 
territory. Many institutions have conducted workload and productivity studies on 
their own campuses (see Diamond and Adam, 1993). In addition, at least four major 
national studies are either ongoing or have recently been completed: the Institutional 
Priorities and Faculty Rewards Project conducted by the Center for Instructional 
Development at Syracuse University (Gray, Froh, and Diamond, 1992; Gray, 
Diamond, and Adam, 1996); the Faculty Survey conducted by the Higher Education 
Research Institute (Dey, Ramirez, Korn, and Astin, 1993); the National Survey of 
Postsecondary Faculty (NSOPF88 and NSOPF93) conducted by the U.S. Department 
of Education (National Center for Education Statistics, 1994); and the National Study 
of Instructional Costs and Productivity conducted by the University of Delaware 
(Middaugh, 1996). These studies are summarized in the full 126-page PICURC study 
report. Expectations and Experiences of Faculty in the 1990s (PICURC, 1996). 

The PICURC Faculty Survey 

To date, there has been significant conceptual work and quantitative research on 
faculty roles, experiences, perceptions, activities, and reward systems. The higher 
education community has responded to public concerns with a determined effort to 
understand the work of its faculty better, to recommend changes to workload and 
reward systems that may be out of sync with good educational practice and the 
expectations of key constituencies, including the faculty themselves, and to educate 
those outside of the academy about the nature and value of aU the work that faculty 
perform. Nonetheless, there continues to be a need for focused research on at least 
four sets of topics. 

1. Student preparation before and after college. An underlying premise of 
higher education and the work of faculty is that students leave college with 
relevant skills enhanced by their undergraduate experience. Past research 
(e.g., the NSOPF and HERI studies) has revealed faculty frustration with the 
quality of students that they have to teach. Periodic reports of declining 
standardized test scores among college applicants, eroding admission 
standards at once-selective institutions, and rising freshman attrition rates, 
further suggest that the level of academic preparation among entering students 
is often inadequate. On the other end of the undergraduate pipeline, 
graduating seniors are often perceived as poorly prepared for post-collegiate 
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life, particularly employment. Anecdotal stories of job placement failures, 
impoverished twenty- and thirty-something college graduates forced to live at 
home with their parents, and the occasional functionally illiterate bachelor’s 
degree holder, fuel this cynicism in the public’s imagination. 

Yet, to what extent are these perceptions valid? Are students taught important 
things in college? What are their skills upon entering college, and most 
important, how adequate are the outcomes? 

2. Faculty teaching activities. In spite of research to the contrary, the belief 
persists that faculty teach too little and that, when they do teach, they do so in 
a manner reminiscent of industrial mass production (May, 1996). The 
PICURC Faculty Survey included a large number of questions — both 
quantitative and qualitative — on teaching activities to assess both the volume 
and the nature of those activities. Of special importance were the 
individualization of instruction, actual teaching loads, use of innovative 
instructional techniques, and the interplay between research and teaching. 

The conflict or complementarity between teaching and research has been at 
the forefront of many debates about the value and productivity of higher 
education. The historical evolution of the American university from an almost 
exclusively teaching-oriented to a heavily research-oriented institution has 
been well documented (Heydinger and Simsek, 1992; Diamond and Adam, 
1993). Such impassive accounts of “how we got here,” however, do little to 
quell the doubts of consumers and critics whose value systems and self- 
interest demand a re-ordering of priorities. The vague societal benefits of 
research, even when substantiated with lengthy lists of beneficial discoveries 
in such practical fields as health and medicine (e.g., Pennsylvania State 
University, 1994), are not considered a sufficient excuse for short-changing 
students and those who help them pay their college bills. 

But are students getting short-changed? That proposition assumes that too 
little teaching is going on and that research does not add value to the teaching- 
learning enterprise. The PICURC Faculty Survey attempted to examine these 
assumptions. 

3. Institutional support of the research and teaching mission. Past research 
suggests that faculty have been asked to do too much with too little — too little 
time, pay, support staff, professional development, and materials. From the 
faculty’s perspective, how valid are these resource concerns, what is their 
perceived impact, and where are the specific pressure points? 

4. The faculty reward system, real and ideal. The reward system for granting 
tenure, promotions, sabbaticals, and annual salary increases has been criticized 
as imbalanced, unrealistic, and insufficiently student-oriented. Some of the 
research reviewed earlier examined this topic from a variety of angles. Of 
primary importance to this study were faculty perceptions of balance within 
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the current system vis-a-vis an ideal one and the locus of control for 
determining faculty rewards. 

The purpose of the present study was to address these questions specifically for the 
independent colleges and universities of Pennsylvania, using the previously cited 
national studies as a backdrop. 



Methodology 

The PICURC faculty survey was designed by staff of the Research Center in 
consultation with faculty, administrative colleagues, and consultants, and 
administered by the participating institutions. The four-page questionnaire, along 
with cover letters from PICURC and institution presidents explaining the purposes 
and procedures of the study, were handed or mailed to faculty on each campus. 
Completed surveys were returned confidentially to PICURC in postage-paid 
envelopes with a unique sequence code on the envelope for tracking purposes. 

Faculty who did not return the first survey were sent a second copy with cover letters. 
A sample of those faculty who did not complete either the original mail survey or the 
follow-up survey were subsequently called by a professional survey research firm and 
administered the survey by phone. Data were collected in the fall and winter of 1995- 
96.. All data were analyzed confidentially by the Research Center. 

The survey was designed for faculty who had been teaching full-time at their 
institution for at least three years. Background questions on the survey permitted the 
Research Center to detect and remove surveys that were inadvertently sent to 
ineligible faculty (fewer than 5% of the completed surveys were thus removed). This 
procedure ensured that all respondents had at least a minimum experience base from 
which to answer the survey questions, The survey had four sections: Students and 
the Undergraduate Instructional Mission; Teaching and Research Activities; 
Instructional Resources and Faculty Reward Systems; and background information on 
respondents. 

Institutional Characteristics 

A total of fifty colleges and universities participated in the Faculty Survey. These 
institutions represent 63% of the 79 members of the Association of Independent 
Colleges and Universities of Pennsylvania who were invited to participate in the 
study. Over half were Baccalaureate colleges; the second most frequent institution 
type was Master’s colleges and universities. There were four Two-year colleges, 
three Doctoral or Research universities, and one Religious college. The overall 
response rate for the survey was 63%, ranging from 34% for the Doctoral and 
Research universities to 86% for the Religious college. Of the 4,912 eligible faculty 
in the sample, 2,493 (51%) returned their completed surveys via mail and 579 (12% 
of original sample, 34% of those contacted by phone) completed the survey by 
telephone. 
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Faculty Characteristics 

Overall, approximately two-thirds of respondents were male; the median age was 50 
years, with a near-normal distribution; almost all (93%) were white; the respondents 
were well-distributed by rank (36% full professor, 34% associate professor, 27% 
assistant professor); most (89%) were tenured or on a tenure track; and almost one- 
fifth were members of a religious order. 

Respondents indicated their primary academic department on an open-ended survey 
question; the responses were then categorized into 14 disciplinary clusters based on 
the Classification of Instructional Programs. The largest cluster was the social 
sciences (16%) and the smallest was a multi-interdisciplinary cluster (5%). Multiple 
specific departments are contained in some clusters, which were employed to reduce 
the disciplinary groupings to a manageable number with robust numbers of 
respondents (or Ns). 

An examination of the higher education backgrounds of responding faculty revealed 
that about two-thirds attended private institutions as undergraduates and almost half 
(48%) attended private institutions for their doctoral studies. Over half (56%) 
attended liberal arts colleges as undergraduates. Significant numbers pursued their 
baccalaureate (44%) and doctoral (34%) studies in Pennsylvania. There were sizable 
differences in the distribution of all faculty characteristics across the different types of 
institutions. 



Survey Results 



Notes on Interpretation 

Tests of statistical significance were not routinely conducted on the data because the 
power of such tests, inflated by the large sample size, yielded differences that were 
statistically significant but trivial in magnitude (see Cohen, 1988). Overreliance on 
significance tests in the social sciences has recently been called into question (Shea, 
1996), further suggesting that they would add little to the largely descriptive analyses 
presented below. For these reasons, multivariate analyses will not be emphasized in 
this report. 

To meet NEAIR brevity requirements, not all of the data collected and analyzed are 
included in this report. Complete data tables, including meaningful breakouts for 
each of the major independent variables in the study, are available in Expectations 
and Experiences of Faculty in the 1990s (PICURC, 1996). 

Part I: Student Preparation 

The first set of questions examined faculty perceptions of the skill levels of incoming 
freshmen and graduating seniors. These questions were based on the view that 
students themselves are an instructional resource: those who are well-prepared for 
college work enrich the instructional setting in ways that underprepared students do 
not. Student skill levels therefore might be expected to influence the nature and 
sophistication of the curriculum itself, the approaches and material that faculty bring 
to their teaching, and ultimately the satisfaction that faculty members derive from 
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their work. Students who systematically fail to meet faculty expectations can have a 
dampening effect on the teaching-learning enterprise. 

The three skill areas where incoming freshmen were perceived, on average, to be least 
well prepared were Writing Skills, Study Skills, and Quantitative Skills (Chart A). 
Fewer than half of the responding faculty had their expectations for Writing and 
Study Skills met by incoming freshmen. The areas where faculty reported the highest 
satisfaction were Computer Skills and Speaking Skills. Computer Skills is an area 
where one might expect generational differences favoring modem student cohorts 
who have grown up using computers for both academic and recreational activities. 



Computer Skills 



Speaking Skills 



Academic Motivation 



Overall Preparation 



Reading Comprehension 



Reasoning Skills 



Quantitative Skills 



Study Skills 



Writing Skills 



Chart A 




Faculty satisfaction with student skill levels was much higher for graduating seniors 
than for incoming freshmen in all areas (Chart B). Almost 90% of survey respondents 
claimed that the overall preparation of most graduating seniors met or exceeded 
faculty expectations and over three-fourths indicated that seniors met or exceeded 
expectations in all specific skill areas except Writing Skills. The three skills areas 
where graduating seniors were perceived, on average, to be least well prepared were 
Writing Skills (72%), Quantitative Skills (77%), and Study Skills (81%) — the same 
three areas for which incoming freshmen received the lowest ratings. Computer 
Skills was again the highest-rated area. 

Part II: Teaching and Research Activities 

The second part of the faculty survey included a series of questions covering both 
institutional and individual faculty behaviors in the areas of research and teaching 
activity. The first question asked whether and how the institution actively encourages 
faculty research. The most frequently cited responses were “Using research 
productivity as one criterion for granting tenure” (79%), “Providing funds for 
research” (62%), and “Maintaining a grants/funded activities offik'V(.6.Q%); ^ T'.' C 
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Chart B 

Faculty Expectations Met/Exceeded by Graduating Seniors 
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The second question asked whether and how the respondent involved undergraduates 
in research. The two most frequently cited methods were “I regularly involve 
undergraduates as co -re searchers” (29%) and “I use undergraduates as subjects in or 
spectators of my research” (17%). Though less frequently, undergraduates are also 
permitted to critique faculty research efforts, particularly at Baccalaureate I colleges 
(17%) and Doctoral and Research universities (1 1%). The latter institutions had 
significantly more evidence of undergraduate involvement in faculty research than did 
the other colleges and universities. 



Chart C 

Ways that Research Improves Teaching 
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In general, the majority (72%) of faculty who do not involve undergraduates in their 
research teach at institutions that do not actively encourage faculty research. 

The third question asked whether and how research improves teaching (Chart C). An 
overwhelming majority (92%) of respondents indicated that their research improves 
their teaching in one or more ways: by keeping the faculty member and course 
content relevant (82%), by providing course content (64%), and by establishing 
credibility for the faculty member’s expertise (63). Significantly, the majority (73%) 
of faculty who indicated that research does not improve their teaching do not involve 
undergraduates in their research. 

The fourth and fifth questions concerned ways in which faculty members attempt to 
meet the educational needs of students on the high and low ends of the academic 
ability spectrum. These items attempted to assess whether faculty individualize their 
instruction to assist students who do not fall in the middle of the talent distribution. 
Attention to individual needs is one of the reputed hallmarks of independent higher 
education, as evidenced by the lower faculty-student ratios and smaller class sizes at 
many private institutions. The results indicated that the vast majority of faculty 
employ a variety of strategies to assist high-ability and low-ability students. The most 
frequently cited strategies for helping high-ability students were “Help them to find 
mentoring and independent study opportunities” (75%), “Have them tutor other 
students” (48%), and “Involve them in my research” (37%). Strategies for helping 
low-ability students included “Refer them to other faculty or qualified staff (66%) 
and “Have them tutored by other students” (64%). Only 8% of faculty indicated that 
they advise high- or low-ability students to transfer to more appropriate institutions, 
reflecting widespread institutional commitment to student retention and success. 
Finally, most faculty engage in more than one support activity: 77% % of faculty do 
two or more things to assist high-ability students, and 73% of faculty do two or more 
things to assist low-ability students. Only 0.3% of faculty do nothing to assist either 
high- or low-ability students. 



The sixth question examined the issues of diversity, innovation, and cross-fertilization 
of content and teaching practices across the curriculum. The frequency of faculty 
responses to each of nine collaborative or otherwise enriching teaching practices is 
presented in Chart D. The most frequently cited example of cross-fertilization was in 
course content, where interdisciplinary and multicultural materials/perspectives were 
relatively common. The next most frequently cited innovative practice was the use of 
computer technology both in and out of the classroom. Use of portfolios and other 
nontraditional evaluation measures to assess student achievement was cited somewhat 
less frequently. Formal team-teaching arrangements were less regularly practiced but 
far from non-existent. They were more likely to occur for a lull term than a partial 
term, and they were more likely to occur within departments than across departments. 
Nevertheless, 51% of respondents did some type of team teaching over a two-year 
period, including 25% across departments. Taken together, these data speak directly 
to the enrichment and integration of the curricula at large numbers and all types of 
institutions and the inclination of faculty to break from tradition in terms of both 
course content and teaching and assessment formats. 
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A final question in this section dealt with faculty teaching loads, specifically, the 
number of sections, independent study students, and credit hours taught by each 
respondent for three consecutive recent (1994-95) quarters/semesters. The results 
indicated an average teaching load of between 2.0 and 3.6 lecture sections per 
semester for those faculty and those semesters when lecture sections were taught; the 
average for lab/discussion sections was between 1 .9 and 3.0 sections per semester and 
included fewer than half as many faculty in the average. Faculty also averaged 
between 1.6 and 2.9 independent study students in those semesters they taught 
independent studies. Refer to the full report for a more complete discussion of 
teaching loads. 



Chart D 

Instructional Practices Used at Least Once in Past Two Years 
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Part III: Instructional Resources 

This section included three sets of items. In the first set, faculty were asked to 
evaluate the adequacy (on a Strongly Agree [+2] to Strongly Disagree [-2] scale) and 
impact of various instructional resources, broadly defined. Just over half of the 
faculty (54%) indicated that their institution provided sufficient time and resources for 
them to be both good teachers and good scholars. Workload demands and budget 
constraints were felt to greater or lesser degrees by faculty according to their 
circumstances. The pressures on female faculty (also documented in the HERI 
survey) and “junior” faculty seemed most acute. 

The next item measured faculty perceptions of another instructional resource 
students themselves— specifically, how institutional emphasis on students’ ability to 
pay affects student quality. This issue is believed to be an important one in private 
higher education in the 1990s, a decade that has witnessed both increasing 
competition for students and budget pressures that have combined to establish ability 
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to pay as a salient student characteristic at many institutions. Results indicated that 
fewer than one-third (30%) of responding faculty considered this a concern. 

The next two items dealt with the impact of budget constraints — again, a major 
concern in private higher education in recent years. Just over half of the respondents 
believed that budget constraints were having significant negative impacts on students’ 
undergraduate experience (53%) and on their own professional experience (58%). 

For the most part, the negative impact was perceived to be stronger on the faculty’s 
professional experience. 

The next set of survey items was directed at faculty who had noted in the previous 
question the significant negative impact of budget constraints on their professional 
experience and asked them to specify the nature of this impact in one or more of four 
areas: on their effectiveness in the classroom or lab, on their effectiveness in other 
professional settings, on their morale or happiness in the workplace, and/or on other 
areas. The vast majority of respondents (92%) identified their effectiveness in the 
classroom or lab as the area of impact; only 1 7% identified their morale or happiness 
as the area of impact. 

The next questions in this section asked those faculty who perceived a significant 
negative impact of budget constraints on their professional effectiveness to rank order 
the importance of specific constraints. In one sense, this question asked those faculty 
who were most severely affected by budget shortcomings to indicate how they might 
allocate funds to one of nine areas, in priority order, to remedy the perceived problem. 
Results indicated that the three highest priorities, in order, were the faculty member’s 
salary, the salaries of other employees and/or staffing levels, and classroom or 
laboratory equipment and supplies. 

Part IV: Faculty Reward Systems 

This section of the survey was divided into two parts. The first posed a series of 
questions about the existence of, and faculty control over, reward systems. The 
second asked faculty to assign weights (adding to 100%) to each of six components of 
the reward system: teaching excellence, research productivity, student advising, 
administrative/govemance work, service to the surrounding community, and service 
to professional organizations. Weights for both the current system on campus and for 
an ideal reward system, as perceived by faculty, were requested. Weights were 
assigned separately for tenure and merit pay reward systems. W eights were 
normalized to total 100% for all responding faculty. 

The first question, “There is an appropriate and reasonable faculty reward system in 
place at this institution,” generally evoked mild to severe disagreement among 
respondents. The second question, “This institution has performance (merit) based 
salary increases,” had widely divergent responses, from mild agreement to extreme 
disagreement. It is of interest that faculty at the same institution differed in their 
responses. At one Baccalaureate I college, for example, 49% of respondents agreed 
with this statement and 51% disagreed with it. While one might imagine this to be an 
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objective, black-and-white question about whether a merit pay system exists on each 
campus, in reality it appeared to be a referendum about not only the existence but also 
the fair implementation of such a system. Token systems that do not consistently 
achieve their objective of rewarding superior performance appear to be recognized as 
such and resented by faculty. A third question, “The faculty at this institution have 
significant control over the nature of the faculty reward system here,” evoked almost 
universal disagreement. Only 7% to 28% of faculty at different types of institutions 
agreed with this statement. This finding highlights an important perceived problem 
the absence or breakdown of shared governance in the definition of the faculty role 
and the reward systems that reinforce it. 

The second set of questions yielded sets of weights for tenure and merit pay systems 
and differences between the current and ideal reward systems. Major differences 
among different types of institutions were striking but not unexpected (Note: To 
conserve space, the charts below represent the response patterns for all faculty 
combined. The narrative describes the more interesting breakdowns by type of 
institution.). At one extreme, for the current system. Two-year college faculty gave 
teaching excellence almost two-thirds of the total weight for tenure and over 50% for 
merit pay; they gave research productivity less than 1% for tenure and less than 2% 
for merit pay, consistent with the faculty’s focus on undergraduate development 
through teaching and advisement during the first two years of college. At the other 
extreme, for the current system. Doctoral and Research university faculty gave 
teaching excellence only 25% and research productivity between 56% and 
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60% of the total weight for tenure and merit pay. It is of interest that faculty at 
Baccalaureate I and Masters I institutions gave teaching excellence more weight than 



research productivity for tenure, but less weight than research productivity for merit 
pay. Student advising received more weight at Two-year, Baccalaureate II and 
Masters II colleges than at other types of institutions; in fact, advising received the 
second highest weight (behind teaching excellence) at Two-year colleges. Service to 
the surrounding community and to professional organizations received relatively low 
weights at all types of institutions. Administrative/govemance work received higher 
weights than student advising at all types of institutions except Two-year colleges. 

With respect to an ideal reward system, for tenure. Two-year college faculty shifted 
weight slightly towards research productivity and service to the surrounding 
community and professional organizations and slightly away from teaching 
excellence, student advising, and administrative/ governance work. Doctoral and 
Research university faculty shifted weights significantly toward teaching excellence, 
slightly toward student advising and service to the surrounding community and 
professional organizations, significantly away from research productivity, and slightly 
away from administrative/govemance work. Every weight shifted slightly for all 
institution types. In general, for all institution types taken together, weights increased 
slightly from the current to an ideal tenure system for teaching excellence, student 
advising, and service to the surrounding community and professional organizations; 
they decreased slightly for research productivity and administrative/govemance work. 

With respect to an ideal reward system, for merit pay, the picture changed 
substantially. Two-year college faculty shifted weights significantly toward teaching 
excellence and significantly away from student advising. These ideal weights closely 
resemble the current weights for tenure at Two-year colleges, suggesting a 
disadvantageous break between the tenure and merit pay reward systems. Significant 
increases fi-om current to ideal systems were also noted toward teaching excellence 
and away from research productivity for 
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Chart F 

Reward System for Merit Pay 
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Baccalaureate I and Doctoral and Research institutions. In general, for all institution 
types taken together, weights increased significantly from the current to an ideal merit 
pay system for teaching excellence and increased slightly for student advising and 
service to the surrounding community and professional organizations; they decreased 
significantly for research productivity and slightly for administrative/govemance 
work. 



Discussion 

The results of the present study suggest that faculty at private colleges and universities 
are actively engaged in significant amounts of teaching that is both pedagogically 
diverse and sensitive to individual student needs. They also participate in research 
programs that benefit undergraduates both directly, through collaboration with 
faculty, and indirectly, through curriculum improvement and faculty development. 
Instructional resources, particularly in the area of salaries and staffing, are less than 
ideal and are perceived to be negatively impacting the experiences of faculty and 
students alike at some institutions. Finally, the current faculty reward system is 
generally considered to place too little emphasis on teaching excellence and top much 
on research and governance activities. Faculty do not believe they exercise sufficient 
influence over the reward systems, a condition that may require serious analysis 
within the college commimity if refinement of those systems is desired. 

In general, faculty appear to be coping well with a variety of stressors. Conflicting 
demands to teach and advise students, perform substantial research, participate in the 
governance of the institution, and perform community and professional service create 
workload and lifestyle conflicts that are difficult to resolve. Compensation for this 
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high volume and wide variety of service is not always believed to be commensurate 
with the quality and quantity of work performed. The effectiveness of the sector’s 
faculty under the difficult budgetary circumstances of this decade, as documented in 
this and other research, is all the more commendable. 

Unfortunately, the resource constraints that higher education has experienced in the 
1990 s may never abate, at least not in the professional lifetimes of those currently 
employed. Faculty at many institutions cannot realistically expect large increases in 
their salaries, staffing levels, and student quality. It is precisely within this context of 
increasing workload demands and stagnant resources that the notion of resource 
reallocation and a refinement of the reward system appear not only desirable but 
imperative if the working conditions of faculty and the educational experience of 
undergraduates are to improve. The literature reviewed earlier includes examples of 
the institutional revitalization that has been accomplished elsewhere under similarly 
trying circumstances. 

All colleges and universities should seek out opportunities to maximize the efficiency 
not only of their faculties but also the systems that define and reward their work. The 
active and thorough involvement of the faculty will be critical to the success of such 
reforms. The leadership of the president and senior staff in engaging the academic 
community to rethink and refine their work and the rewards they receive for 
performing it will be critical. 
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PLACEMENT TEST SCORES AND STUDENT PERSISTENCE: 
INSTITUTIONAL CONSIDERATIONS FOR ABILITY TO BENEFIT 



Alan J. Sturtz 

Director, Institutional Research, Planning & Assessment 
Gateway Community-Technical College 



Introduction 

Long before Ability to Benefit (ATB) was a popular Department of Education 
buzzword, community colleges recognized its value for the large number of 
educationally "disenfranchised" in their service areas. Currently, however, while 
ATB may well stand as important as ever, institutional and state-level policy 
decisions regarding allocation of resources (faculty as well as classrooms) must be 
made only with hard data: Although a college has an open admissions policy, its 
individual programs are not precluded from requiring higher academic admission 
standards; moreover, it appears this tenet can now be applied to pre-college 
(developmental) courses as well. 

The question then becomes, can all students benefit from the courses offered 
at the community college? The issue raised by this question are as follows: can the 
college tell potential students that the scores on their required English placement 
tests' indicate that they lack not only the appropriate skills to proceed from a 
developmental into a college-level English course but also the basic requisite skills to 
enroll and succeed in most other courses at the college? If not, will sufficient 

' The college requires all new (first-time at the college), full-time students to take placement tests in 
English usage/writing, reading and mathematics; part-time and non-degree students who wish to take English < 
mathematics also must take the placement test(sf The test(s) is waived for any student who has successfully 
completed college-level English or mathematics at another college or university. 
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resources be allotted for the college then develop and offer additional lower-level pre- 
college courses to meet these needsj or, with no new resources, will the college cut 
back on elective courses to address the growing trend of un^ and under-preparedness? 

Data Analysis 

To assess the relationship between the scores on the placement test and 
success in developmental writing (ENG 100) and also student persistence rates, the 
director of institutional research, planning and assessment and the coordinator for 
developmental writing at Gateway Community-Technical College initiated a study of 
students who took the placement test between 1992 and 1995, who were then 
recommended to take ENG 100, and who actually enrolled and registered for the 
course. During this time frame, almost 5,000 students took the English section of the 
placement test, over 3,000 were recommended for developmental writing and 2,689 
actually enrolled in the course.^ 

The key element in the analysis will be the score on the placement test; the 

hypothesis proposes that students who test below a given level will not receive 

3 

satisfactory grades in English 100 and will subsequently leave the institution. 

The possible range of scores on the Essay section of the placement test is 0 to 12, with 
6 or below a solid recommendation for the developmental writing course (hereafter 
ENG 100). A score of 7 or 8 is a borderline recommendation for either ENG 100 or 
the college-level English Composition course, depending on the score on the grammar 

^ The actual number of students enrolled in ENG 100 since 1992 was 2,866; 163 who enrolled without 
having the recommendation and 14 students who first enrolled in Fall 1996 were not included in the study. 

^ Other intervening variables, of course, exist that can cause students to leave or to re-enroll in a 
subsequent semester. 
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section of the test. A score above 8 is usually a recommendation for the composition 
course. For the study population, about one-quarter (25.8%) scored 4 or below; 17 
percent scored 5, 40 percent scored 6 and 17 percent in the 7 to 8 range. It should be 
noted that 37 students who had recommendations to enroll in the composition course 
chose to enroll in the developmental course. 

A comparison with the overall college population shows similar numbers by 
sex (female:male is 57%:43% in ENG 100, 60%:40% overall) but a higher percentage 
of minority students (African-American 27% in ENG 100, 17% overall; Hispanic 
10% in ENG 100, 8% overall; the percentage of Asian- American students is less in 
ENG 100 than overall— 1.6% to 2.9%). Whites are about two-thirds (65%) of the 
college population but slightly more than half (56%) of the study population. 



TABLE 1. Distribution of Test Scores by Sex 



FEMALES 


MALES 


SCORE 


■ N 
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0-2 


32 


2.2 


58 


5.1 
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38 


2.5 


55 


4.8 


4 


252 


16.4 


260 


22.5 


5 


245 


16.0 


209 


18.1 


6 


652 


42.6 


416 


36.0 


7 


177 


11.6 


94 . 


8.1 


8 


119 


7.8 


57 


4.9 


9-12 


17 


1.1 


8 


0.7 


ALL 


1,532 


57.0 


1,157 


43.0 



Looking at the range of scores by sex, 32.4 percent of the males, compared to 
21 percent of the females scored 4 or below; 18 percent of the males compared to 16 
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percent of the females scored 5 and 36 percent of the males compared to 42.6 percent 
of the females scored 6. Females had a higher percentage representation in the 7-8 
range as well (19.4 % to 13%). By ethnicity, there was almost an even comparison 
among the four groups scoring 6 or below (83% White, 80% African-American, 83% 
Hispanic, and 88% Asian- American). At a score of 4 or below the respective 
percentages are 22.6%, 28.3%, 28.3%, and 73.4%). 



TABLE 2. Percent Distribution of Test Scores by Ethnic Group 



SCORE 


ETHNIC GROUP 


WHITE 


AFRICAN- 

AMER. 


HISPANIC 


ASIAN- 

AMER. 


AMER- 

INDIAN 


NO 
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0-2 


2.7 


3.6 


3.5 


13.9 


0.0 


5.6 


3 


3.0 


4.4 


3.8 


9.3 


0.0 


1.6 


4 


16.9 


20.3 


21.0 


30.2 


50.0 


28.2 


5 


17.1 


16.2 


17.2 


18.6 


16.7 


16.9 


6 


42.8 


36.0 


37.4 


16.3 


33.3 


37.1 


7 


9.2 


12.3 


11.1 


11.6 


0.0 


5.6 


8 


7.2 


6.5 


5.7 


0.0 


0.0 


3.2 


9-12 


1.2 


0.6 


0.4 


0.0 


0.0 


1.6 


ALL 


56.5 


27.3 


9.7 


1.6 


0.2 


4.6 



About 1 1 percent of the students were enrolled at the college prior to taking 
the placement test, most of them (6.0%) less than four semesters. Of the remainder. 
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18.4 percent began during the 1992-93 academic year, 22.7 percent during 1993-94, 
28.2 percent during 1994-95 and 20.1 percent during 1995-96. Regarding persistence, 
of the entire sample, 127 (4.7%) completed degree or certificate requirements and 793 
(29.5%) are still enrolled (Fall 1996). These next sections will analyze credits, QPA 
and length of enrollment in the aggregate and by essay score; all results reflect Fall 
1996 enrollment. 

Four hundred of the almost 2700 students in the study (14.9%) earned zero 
academic credits; 43 percent earned 15 credits of less, 22.6 percent earned 16-30 
credits, another 8.8% earned 31-45 credits and the remaining 10 percent (287) earned 
more than 46 credits— including, as mentioned above, the 127 who graduated. An 
analysis of number of credits earned by test score is shown below. 

While it appears obvious that those students with higher scores tend to 
accumulate more credits— persistence— policy and effectiveness issues must deal with 
the 15 percent from all score levels accumulating zero credits, the additional nine 
percent accumulating three credits or less and the 10.5% more with six credits or less. 

An analysis of QPA shows that among the population in this study, more than 
40 percent (42.8%) had a grade average of less than 2.0 on a four-point scale, about 
one-third were in the C range and about one in five (23%) had a B or higher. Looking 
at QPA by essay score confirms the findings of the credit/essay analysis:, over 50 
percent of those receiving a score equal to or less than 2 had a QPA below 2.0. This 
same QPA range was found among 56 percent of those scoring a 3 and 50 percent of 
those scoring a 4. Of those scoring 5, 6 or 8 on the test, 43%, 38% and 31%, 
respectively, had QPAs below 2.0 Uncharacteristic of this declining trend, almost 
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half (48.8%) of those scoring 7 had QPAs below 2.0. 



TABLES. Percent Distribution of Credits Earned by Test Score 



SCORE 
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CREDITS EARNED & PERCENT 
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17.6 


46.5 


21.6 


6.1 


8.2 


15 
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454 


16.7 


46.2 


20.7 


9.0 


15.3 


14 
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1068 


12.0 


41.9 


23.6 


9.3 


13.4 


64 


7 


271 


13.6 


39.2 


17.9 


10.2 


12.5 


17 
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176 


13.6 


39.2 


24.9 


10.2 


11.9 


9 


9-12 


25 


4.0 


40.0 


32.0 


20.0 


4.0 


1 


ALL 


2689 


14.9 


43.4 


22.2 


8.8 


10.7 


127 



The issue of persistence is examined in a matrix of first-time and last-term 
enrolled, further refined, as with the other variables, by test score. Of the entire 
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population, an aggregated 25 percent (678 students) left the college after one term; an 
additional 16% left after two terms. Perhaps surprisingly, this favorably compares to 
a one-term attrition rate of almost 40 percent for the college as a whole. On the 
positive side, 793 students of the original 2,689 (29.5%) are still enrolled in Fall 
1996. A profile of the students who left after one term shows a female to male ratio 
of 56 percent to 44 



TABLE 4. Distribution of Test Scores by QPA 
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8 
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2 . 


2 


5 


11 


2 


2 


0 



percent; 49 percent White, 34 percent African-American, 10 percent Hispanic and two 
percent Asian-American; about one -third (30.4%) scored 4 or below on the placement 
test, 70 percent earned three or fewer credits (50% earned zero) and 76 percent had a 
QPA below 2.0 (60% below 1 .0). 

IerJc 
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Slightly more students are currently enrolled that left after one semester (793 
to 678, about 4.5 percent). A profile of these students shows a female to male ratio of 
56 percent to 44 percent; 64 percent White, 20 percent African-American, eight 
percent Hispanic and two percent Asian-American; Twenty-eight percent of this 
group scored 4 or below on the placement test (another 18.3% scored 5 and an 
additional 40 percent scored 6). QPA distribution for the persisters shows less than 
20 percent below 2.0, 45 percent in the 2.0 to 2.9 range and 35.7 percent over 3.0 (12 
students had a perfect 4.0 compared to seven among the leavers). For duration of 
enrollment, about one-third (3 1 .3%) began their studies at the college more than five 
semesters ago (Fall 1996 begins their third year); one-fourth began in Fall 1994 and 



30 percent began in Fall 1995. 

TABLE 5. Two-Term Attrition Rate by Test Score 
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Compared with those who left after one semester, there is no difference in the 
female to male ratio. While about half of those who left after one term where White, 
almost two-thirds (64%) of those who persisted were White; one-third of first 
semester leavers were African American, compared to one in five persisters. 
Percentages for both Hispanic and Asian-American students in the leaver vs. persister 
groups are similar (10% to 8% and 2% to 2%, respectively). Distribution of test 
scores showed 30 percent of first-term leavers scoring 4 or lower compared to 28 
percent of persisters. Almost half the leavers scored 5 and one-third scored 6 
compared, respectively, to 1 8 percent and 40 percent among the persisters. While 1 4 
percent of persisters scored 7 or higher, about 1 8 percent of leavers scored 7 or 
higher. From this analysis, it would appear that there is no difference between the 
two groups regarding test scores, except in the 5 and 6 score range. 

Unlike the baccalaureate institutions, most community colleges do not 
measure success solely in terms of graduates. While only about half of any years s 
graduates at Gateway Community-Technical college completed their programs of 
study in five years or less, only about 1 0 percent of all enrolled students graduate. 
Following proposed NCES guidelines, for the first-time full-time degree seeking 
cohort in fall 1992, 15.2 percent graduated by spring 1995. Only 4.7 percent of 
students the students who were recommended for ENG 100 in this study graduated: 
however, over 90 percent graduated in four years (eight semesters) or less-including 
16 certificate graduates. The female to male rations was 57 percent to 43 percent, 
almost identical to the other subgroups analyzed in this study. Whites, African- 
Americans and Asian- Americans were more highly represented in this population 



than in the college as a whole; only Whites and Asian-Americans were more highly 
represented in the graduate population than in the population recommended for 
developmental writing. Half the graduates scored 6 on the placement test; 21 percent 
scored higher. Eleven percent scored 5 on the test and 17 percent scored lower. 
Conclusion and Recommendations 

While there is no definitive score below which a student will receive an 
unsatisfactory grade in developmental writing and subsequently leave the college, 
students scoring 4 or below on the English placement test should be considered higher 
risk. These students tend to be male (32.4% vs. 21.1% female) and minority (28.8% 
vs. 22.6% White; with Asian American representation of over 50%). They are more 
likely to leave the college after one or two semesters without having completed, with 
a grade of D or higher, any course. 

While this analysis should not be the sole factor in determining ability to 
benefit— or allocation of institutional resources— it does provide vital information for 
reconsideration of academic policies and support services for strengthened academic 
advising, alternative modes of instruction, developmental course scheduling, and 
early intervention programs. 





IN SEARCH OF PEER INSTITUTIONS: TWO METHODS OF EXPLORING AND 
DETERMINING PEER INSTITUTIONS 
Bruce P. Szelest 

Associate for Institutional Research 
State University of New York at Albany 



Introduction 

The State University of New York (SUNY) at Albany, one of 64 campuses in 
the state university system, recently undertook an exercise to develop a list of peer 
institutions that could be used in a system-wide planning processes. The backdrop for 
this endeavor was the decision of the state university system to undergo a resource 
allocation review of its campus funding formulae. One component of that review is 
to identify similar colleges and universities and examine their requisite funding and 
^^ormance across ohosen variables. Peer groups are not necessarily to be used in 
creating funding algorithms, though, but more so to post facto validate funding 
patterns that might emerge from collective discussion. 

Exactly which variables are to be used as performance or financial measures 
are not yet formally determined. Our institution was asked to develop a set of peer 
institutions from which measures could later be compared. Rather than having a 
specific set of peer institutions externally forced upon us, we were afforded the 
opportunity to choose our own comparison institutions. We therefore needed to 
develop a peer institution selection strategy that encompassed a variety of potential 
variables for later use. This paper outlines the underlying rationale for this 
undertaking, describes the two analytical methods used, and identifies the peer 
institutions. 

Theoretical Framework 

Almost two decades ago, Terenzini et al. (1980) stated that “in recent years, 
planners, administrators and researchers have increasingly sought improved ways to 
classify higher educational institutions for descriptive, comparative, and analytical 
purposes.” Sixteen years later we face a similar situation. Teeter and Brinkman 
(1992) forewarned that funding levels and institutional prestige can be directly or 
indirectly affected by the use of peer groups. As noted earlier, one of the stated 
purposes for undertaking this study is to develop a peer group that will certainly be 
used in the assessment of funding levels and institutional prestige through subsequent 
performance measures. 

There is no one generally accepted standard technique with which to identify 
peer institutions. Teeter and Brinkman (1992) suggest that in order to ensure success 
in not only developing a set of peer institutions, but in actually implementing the use 
of a peer group for comparative purposes, the analyst needs to assess the political 
environment he or she operates within. A political dynamic can indeed be seen in 
how comparison groups are viewed. They point out four major types of comparison 
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groups; 1) competitors, with regard to applicants, faculty, or financial resources, 2) 
aspirational, those institutions we strive to be like in some respects, 3) predetermined, 
those institutions that are natural, traditional, or which share jurisdictional area, and 4) 
peers, which can be used in benchmarking. 

SUNY Albany certainly competes for students, more so at the graduate level, 
with other major public research universities, and to a certain extent, we compete with 
the other SUNY university centers for undergraduate students and financial resources 
as well. If a peer group is to be used to assess financial support, it is certainly in our 
campus’s best interests to include some institutions whose level of state support we 
aspire to emulate. While we would prefer to have the other three SUNY university 
centers in the comparison group based on a common research mission within the 
same state system, simply choosing them on a predetennined basis without regard to 
quantitative analytical justifications would not have most likely been politically 
acceptable to the system-wide administration, nor, as well, to the other university 
centers. 



Brinkman and Teeter (1987) summarize cluster analysis, hybrid, threshold, 
and panel review approaches that can be thought of as ranging from purely data 
driven for cluster analysis, to largely subjective judgment where panel reviews are 
concerned. This research endeavor, statistically driven with respect to peer institution 
selection, employed a number of subjective judgments in regard to variable selection, 
implicit weighting schemes, and also in the handling of missing data. In any analysis, 
while we may seek objectivity, implicit and explicit subjective judgments ultimately 
play some role. Hopefully, we can recognize them when they occur. 

Dataset Development 

As part of ongoing activities, the Albany Office of Institutional Research has 
over the past several years collected data and sought out sources of information to 
contribute to a dataset capable of producing institutional profiles across a wide array 
of variables for use in ad hoc studies and environmental scanning. Data are compiled 
from IPEDS', the National Research Council’s (NRC) 1995 report on Doctoral 
Education, the Chronicle of Higher Education, college guide books, and two soon-to- 
be published research endeavors. 

A dataset was developed that included 1 04 public Doctoral I and Research I & 
II universities. A decision was made early in the process to only consider those 
institutions with a strong emphasis on doctoral education, hence Doctoral II and those 
institutions that do not offer the doctorate are excluded from analysis. Potential 
institutions are also restricted to the public ranks due to public control of the SUNY 
system. While Albany does not have a medical or engineering school, those 
institutions that do are included in the analysis. It was felt that even though Albany 



' IPEDS data were obtained from the NSF sponsored CASPAR system developed by Quantum 
Research Corporation. Visit Quantum’s web page www.qrc.com for additional information and access 
to the CASPAR system. 




126 



127 



does not offer medical or engineering curriculums, we are indeed similar to these 
institutions in other respects. As the analyses will show, several institutions in these 
categories are indeed identified as potential peers. 

Variable Selection 

After mission-based narrowing that excluded Doctoral II, Comprehensive, 
Liberal Arts, Two-year and Professional schools, variables were selected for study 
based on their appropriateness in capturing dimensions of finance, size, complexity, 
and quality. The final measures chosen are more heavily biased in favor of financial 
and quality considerations. Owing to the political considerations cited earlier, 
fourteen financial, eleven quality, seven size, and six complexity measures were used 
in the final analysis. It is important to note that individual measures selected for 
inclusion were made in conjunction with campus decision makers and reflect their 
concerns. 

Several IPEDS financial variables can be considered to address both overall 
support and more specific expenditure functionality. Total current funds revenue per 
student and educational and general expenditures per student are used to gauge the 
overall expenditure patterns at potential peer institutions. Specific expenditure 
account categories of student services, research, library, and administrative 
expenditures are also examined on a per student basis. The expenditure accounts of 
instruction and academic support are combined before conversion to a per student 
basis out of concern that institutions are more apt to co-mingle or cross-classify these 
expenditure types. The multi-year average research expenditure per full-time faculty 
member, published by the National Research Council (NRC), is used as a long term 
indicator of research expenditures with respect to faculty inputs. Average full-time 
faculty salaries in 1994, published by The American Association of University 
Professors (AAUP), are used to represent faculty financial support. Average full 
professor salary in 1994, also obtained from AAUP, is included in the analysis to 
provide a measure of financial support for more senior faculty . Because an impetus 
for conducting interinstitutional comparisons in the first place concerned the 
allocation of state resources to SUNY campuses, IPEDS sourced state appropriations 
and non-state support (endowment and gifts) per student measures are included in the 
analyses. Finally, tuition and fee revenue per student, also obtained from IPEDS, is 
used to measure the financial aspects of the institution which students are required to 
contribute to. 

Almost half (five of twelve) of the quality measures used are obtained from 
Graham and Diamond’s (1996) doctoral university classification which is grounded in 
faculty research achievement. Four individual measures of faculty research activity 
are used. Total publications over a multi-year period, the number of publications per 
full-time faculty member, the total number of citations in major humanities, social 
science, and science journals, and the citations per full-time faculty member are used 
to gauge both overall faculty research activity as well as publishing productivity of an 
individual institution’s faculty. An overall composite of faculty quality, derived from 



127 



128 



these measures by Volkwein and Malik (1996) is used as a measure of overall faculty 
research quality. Average faculty reputational quality is calculated for each institution 
by computing the mean of each institution’s NRC (1995) faculty quality rating for 
rated programs. Because low student faculty ratios are often looked to as being more 
amenable to a quality education, a student faculty ratio is computed by dividing 
IPEDS derived student FTE by the number of full-time faculty reported in IPEDS. 
More commonly accepted indicators of student quality are the freshmen acceptance 
rate, percent of freshmen returning for their sophomore year, the six-year graduation 
rate, and the percent of alumni that contribute to the institution. These measures are 
taken from U.S. News, one of the major players in the collection and publication of 
this type of information. Unlike most guidebooks, U.S. News asks institutions to 
certify their data after initial compilation by U.S. News staff. 

The measures of institution size used in this research are those traditionally 
cited in the literature as being synonymous with defining size in higher education 
institutions. Total student FTE and headcount enrollment, as well as graduate 
headcount are used to capture both size and mission. Headcount enrollment is 
reported by IPEDS and FTE students are calculated by adding full-time headcount 
and one-third of the part-time headcount enrollment. The total number of all degrees 
awarded, as well as the number of doctoral degrees awarded are an attempt to 
measure one aspect of institutional output, graduates, and are also taken from IPEDS. 
IPEDS also contain the number of full-time faculty in 1994. Finally, the NRC (1995) 
has published the total number of volumes and serials in each doctoral institution’s 
library. 



The five complexity measures address administrative complexity, student 
body composition, and graduate emphasis. Volkwein and Malik (1996) garnered the 
total number of deans and vice presidents at each institution from the Higher 
Education Directory and added them together to form a measure of administrative 
complexity. The percentage of the student body that is minority is calculated from 
IPEDS enrollment data, as are the percentage of total enrollment and degrees awarded 
that are post-baccalaureate. Lastly, the percentage of students living in college 
dormitories is taken from the College Board survey. 

Missing Data 

In any undertaking using such disparate data sources missing data is a valid 
and strong concern. Restricting the potential peer pool to public Research I & II, and 
Doctoral I institutions reduced the dataset to 104 institutions. Only selected 
institutions were included in the NRC study of doctoral programs, and fifteen of the 
public Doctoral I institutions are not included. These fifteen institutions are used in 
the analysis in determining the percentile ranks discussed below, but are not classified 
in the peer analyses due to the missing data. Preliminary analyses that did not utilize 
NRC data failed to group these institutions significantly close enough to Albany to 
justify the time and effort needed to obtain their data. 
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Several institutions were found to have missing IPEDS data though. Most 
often this resulted from FICE code mismatches. Because smaller, more compact 
database files containing the (alphanumeric) names of each institution were created 
from each of our major data sources, it was a trivial matter to identify the missing 
institutions, correct the FICE codes used in matching the files, and update the main 
database. An exception is the State University of New Jersey which reported all of 
their 1993 IPEDS financial data for the total system, rather than for each of the 
respective campuses. In order to include the Rutgers-New Brunswick campus, which 
Albany views as a potential peer based on its reputation and perceived similarity, 
seventy-eight percent of the financial figures were imputed to the New Brunswick 
campus as an estimate based on the fact that seventy-eight percent of the system 
enrollment was reported for the New Brunswick campus. 

Methodology 

After considerable preliminary examination of institutions by more 
conventional schemes such as Carnegie status, the presence or absence of medical or 
engineering schools, as well as by various demographic and ecological characteristics, 
two statistical techniques are used to select peer institutions. The rank distance 
method uses a methodology recently learned from Carol Berthold, University of 
Maryland System, at the 1996 AIR Forum in Albuquerque, NM^. This method is very 
similar to that used by the University of Kansas and described by Teeter and Christall 
(1987), but uses a percentile rank order of institutions on each measure rather than Z 
scores to calculate similarity/dissimilarity. The cluster analysis method uses the 
outputs of a principal components analysis to group universities into similar clusters. 
Terenzini et al. (1980) were amongst the early pioneers of this technique at Albany in 
the late 1970s, and the time seemed ripe to revisit their work in the mid 1990s. 

The rank distance method calculates the distance of each public university 
from Albany using each institution’s percentile rank across measures. First, an 
institution’s percentile rank within the dataset is calculated for each measure. Second, 
for each measure the difference between Albany’s (the reference institution) 
percentile rank and the percentile rank of every other institution is calculated. Third, 
the differences are squared and summed for each institution. The square-root of the 
summed distance measure then becomes a composite measure on which each 
institution’s distance from Albany is based. This is a standard Euclidean distance 
calculation, and the closer each university is to Albany’s percentile rank on each 
variable, the more likely it will be listed as a nearby peer. Each of the thirty-eight 
measures in the analysis receives equal weight, and complete data is required for each 
case to calculate a distance score. 

The Kansas classification described by Teeter and Christall (1987) utilized a 
weighting scheme to elevate the importance of certain variables after standardization. 



^ The author is also indebted to Ms. Berthold for her insights on making use of national databases such 
as IPEDS and NSF sourced data, as well as for her references to the CASPAR system .which greatly 
facilitated this research. 
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While this analysis does not use explicit weights for the variables chosen, it should be 
noted that an implicit weighting scheme is active in that fourteen financial, eleven 
quality, eight size, and five complexity measures are used. Hence, elements of 
finance and quality are more dominant in assessing institutional 
similarity/dissimilarity with the reference institution. 

The factor and cluster Analysis technique uses principle components analysis 
(varimax rotation, and Kaiser’s criterion for Eigenvalue selection) to reduce the thirty- 
eight variables to seven factors that reflect dimensions of funding, size, quality, 
complexity, and research and graduate mission. The factor scores are exported and 
used in a cluster analysis. Institutional clustering via a dendogram can then be 
examined to determine how institutions cluster with the reference institution. 

Each of the factors receives equal weight in building the clusters, and 
complete data is required for each university. Once again, a conscious decision was 
made not to weight the factors, even though no design induced implicit weighting 
scheme existed. The analysis calculates a standard Euclidean distance measure for 
each institution based on the standardized factor scores, and uses the “complete 
linkage” hierarchical agglomerative technique to group institutions into relatively 
homogeneous clusters. The clusters are formed based on the minimum maximum 
distance score between institutions, which is compared at each successive step until 
the researcher decides to stop cluster formation (based on professional judgment 
about diminishing returns). Each university’s cluster can be as small as two 
campuses, or can be built larger to incorporate larger numbers, depending on the 
clustering algorithm and the researcher’s objective. 

The underlying purpose of this analysis is to identify those ten to twenty 
institutions that group closest to Albany. This is easily accomplished by using a 
dendogram to trace out Albany’s clustering pattern to successive institutions. By 
definition, similarity between institutions becomes less distinct as the clusters grow. 

It is also simply a matter of visual inspection to determine where in the clustering 
process those institutions one may be predisposed to as peers actually enter into 
Albany’s cluster. 

Results 

Univariate statistics for the thirty-eight measures used are reported in 
Appendix A. Even though only public doctoral granting universities are included in 
the analysis, brief review of the means and standard deviations suggests a great deal 
of variability among them, and further reinforces the need for a comparison strategy. 
The financial measures exhibited the most variability, followed by the size measures 
and the quality measures. There was considerably less variation in the complexity 
measures, suggesting either that this concept may be more difficult to measure or that 
these universities are indeed more alike in this respect. 

The bivariate results reported in the correlation matrix in Appendix B indicate 
measures, in general, that are highly correlated with each other. For example, total 
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current funds revenue per student has a pearson’s correlation coefficient above 0.50 
with every financial measure except for student services expenditures per student (r = 
0.33) and average full professor’s salary (r = 0.47). Furthermore, it is also highly 
correlated with percent of enrollment that is graduate, average faculty reputation, the 
faculty quality measure, total and per faculty publications, and also with citations per 
faculty member. Other measures were also highly correlated both within and across 
the hypothetical dimensions of funding, size, quality, and complexity. While more in- 
depth discussion of these relationships is not entertained here due to space limitations, 
this does though suggest that a factor analytic technique that controls for possible 
multicolinearity may be more suitable. Campus decision makers, though, did view 
the rank distance method as beneficial with respect to evaluating institutions across 
these measures. 



Table 1 below shows the nineteen institutions that have the closest composite 
rank distance from Albany. Overall, the range of distance scores, range from 0 for 
Albany to 2.88 for the furthest institution, the University of Minnesota (not shown). 
The overall magnitude of the distance scores is a function the number of measures 
used. The five closest institutions to Albany are SUNY Binghamton, U Connecticut, 
U Oregon, SUNY Buffalo, and Georgia Tech. Delaware, UC Santa Barbara, and U 
South Carolina tie as the next most similar institution with a distance score of 1 .70. 
Campus decision makers can now use this table to examine not only how close 
institutions are overall to Albany, but they can also see how Albany’s percentile rank 
for different measures compares with those of other institutions. Due to space 
limitations, columns (1) through (1 1) in Table 1 only show the institutional percentile 
rank of selected measures. 



Tible I 



Albany's Top 20: Rank-Distance Analysis 
Percentile Rank within All Public Research I & M, Doctoral I Universities (N=104) 



Selected Measures Only ^ — 

Financial I Size 1 Complexity I Quality 



Composite 
Rank Distance 
From Albany pei 


Total Instruct & Acad State Avg Student 

Revenue Suppt Expend Apptop Full Prof FTE Full-time Pet Min Pet Grad Avg 

Student per Student per Student Salarv Enroll Faculty Students Enroll Fac Rep 

m (2) (3) (4)' (5) (6) (7) (8) (9) 


Grad Stu/Fac 
Rale Ratio 

(1°) (II) 


SUNY Albany 


0.00 


34.6 


53.9 


46.2 


90.8 


23.1 


19.2 


73.1 


68.3 


46.1 


86.1 


62.5 


SUNY Bingham ion 


1.47 


26.9 


31.7 


51.9 


85.4 


12.5 


6.7 


73.1 


26.9 


21.4 


93.8 


75.0 


U Conn 


1.58 


48.1 


63.5 


44.2 


92.2 


48.1 


64.4 


41.4 


83.7 


49.4 


80.3 


26.0 


U Oregon 


1.64 


29.8 


33.7 


18.3 


36.9 


26.9 


23.1 


36.1 


48.1 


56.2 


51.9 


74.0 


SUNY Buffalo 


1.65 


54.8 


73.1 


85.6 


97.1 


62.5 


59.6 


64.9 


87.5 


53.9 


44.2 


61.5 


Georgia Tech 


1.68 


77.9 


67.3 


80.8 


77.7 


17.3 


18.3 


76.0 


80.8 


67.4 


80.3 


37.5 


Delaware 


1.70 


56.7 


70.2 


6,7 


87.4 


41.4 


50.0 


21.2 


8.7 


62.9 


87.5 


46.2 


UC Santa Barbara 


1.70 


42.3 


45.2 


49.0 


94.2 


42.3 


26.0 


89.4 


4.8 


77.5 


77.4 


94.2 


U So. Carolina 


1.70 


31.7 


65.4 


48.1 


26.2 


58.7 


62.5 


73.1 


94.2 


36.0 


63.5 


43.3 


Virginia PoK lech 


1.73 


55.8 


47.1 


42.3 


62.1 


78.9 


87.5 


45.7 


58.7 


58.4 


86.1 


96 


UC Santa Cm/. 


1.74 


51.9 


64.4 


62.5 


83.5 


8.7 


3.9 


89.4 


2.9 


57.3 


58.7 


80.8 


William & Mary 


1.76 


36.5 


52.9 


12.5 


W).() 


3.9 


8.7 


51.4 


76.0 


69.7 


99.0 


3.9 


Cicmson U 


1.79 


G4.4 


34.6 


71.2 


39.8 


26.0 


39.4 


29.3 


41.4 


25.8 


84.1 


30.8 


Virginia Commonw 


1.79 


92.3 


82.7 


36.5 


48.5 


34.6 


35.1 


78.9 


78.9 


25.8 


260 


51.9 


U Rhode Island 


1.79 


39.4 


42.3 


22.1 


57.8 


14.4 


21.2 


8.2 


34.6 


23.0 


71.2 


24.0 


Oregon State U 


1.81 


75.0 


56.7 


76.9 


35.4 


21.2 


17.3 


45.7 


45.2 


55.1 


47.1 


58.7 


U Mass-Amherst 


1.82 


65.4 


62.5 


52.9 


65.1 


59.6 


72.1 


36.1 


33.7 


71.9 


73.1 


21.2 


Washington State 


1.83 


66.4 


69.2 


78.9 


31.6 


38.5 


55.3 


29.3 


13.5 


37.6 


55.8 


26.9 


SUNY Stony Brook 


1.89 


93.3 


85.6 


94.2 


96.1 


24.0 


40.9 


87.5 


91.4 


78.7 


47.1 


27.9 


U Virginia 


1.91 


95.2 


86.5 


33.7 


86.4 


46.2 


57.2 


70.2 


98.1 


87.6 


100.0 


35.6 



With this table, a campus decision maker can see that Albany is in the forty- 
sixth percentile with respect to state appropriations per student (column 3). The table 
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also shows that the other SUNY university centers at Binghamton, Buffalo, and Stony 
Brook fare better on this measure with percentile ranks of 5 1 .9, 85.6, and 94.2 
respectively. It can also be observed that some institutions on this “top 20” list of 
similar institutions receive even less state appropriations per student than Albany 
does, such as U Oregon, Delaware, William and Mary, Virgina Commonwealth, U 
Rhode Island, and U Virginia. Obviously, the Albany administration would rather 
have their per student state funding more on par with the other SUNY university 
centers than with the lower funded institutions. While securing more comparable 
funding within the SUNY system is a political matter beyond the purview of this 
research, a major advantage offered by the rank distance method is the ability to 
compare peer group institutions within a broader framework that also includes non- 
peer institutions. 

The second method used to develop a set of peer institutions is a factor and 
cluster analysis technique. When the thirty-eight measures described above are 
subjected to principal components analysis, seven dimensions or factors emerge 
which explain eighty-three percent of the total variance. Table 2 shows the factor 
loadings and the resulting dimensions. 



Table 2 







General 


UG 


Graduate 


Stu/ Fac Financial 




Si/e 


Financial 


Quality Complexity Emphasis 


Ratio Support 


Tot FTE enroll 


0.98 










Tot degs awarded 


0.95 










Full-time faculty 


0.93 










Total enroll 


0.93 










Numb doct degs awarded 


0.87 










Full-time grad students 


0.8S 










Tot vol & serials in library 


0.81 










Numb NRC doct progrs 


0.78 










Faculty publications 


0.75 


0.52 








Fac pubs in most freq journals 


0.69 


0.47 








Administrative complexity 


0.64 








-0.43 


Avg fac reputation 


0.63 


0.49 








Resch exp per student 




0.90 








Fed & state grants per student 




0.90 








E&G exp per student 




0.87 








Resch exp per faculty 




0.84 








Tot revenue per student 




0.80 








Endow & Gib rev per student 




0.73 








Faculty quality composite 


0,54 


0.69 








Publicatios per faculty 




0.67 




0.42 




Instr & acad supt exp per stu 




0.66 




0.47 




Admin exp per student 




0.58 








Citations per faculty] 




0.55 




0.49 


0.45 


Six-yr grad rate 






0.87 






Frosh retention 






0.82 






Pet ug live in dorm 






0.77 






Alum give rate 






0.60 


-0.53 




Frosh accept rate 






-0.55 






Avg fac salary 


0.43 




0.45 


0.42 




Pet minority students 








0.76 




Stu serv exp per student 








0.68 




Avg prof sal/ 1 (XX) 






0.45 


0.55 




Lib exp per student 




0.52 




0.52 




Pet degs awarded doct 








0.93 




Pet grad enrollment 








0.79 




Student/faculty ratio 










0.75 


Tuition & fee rev per student 










0.86 


State approp per student 




0,54 






-0.54 


Total Variance Explained 


43.6% 


13.0% 


9.1% 


6.3% 4,1% 


3.6% 3.3% 
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These dimensions suggest a further delineation of the four general dimensions 
hypothesized earlier. The dimensions of size and general finances combine to explain 
56.6 percent of the total variance. The fact that faculty quality, publications, citations, 
and average reputation load on the general financial dimension is an interesting 
finding; and points out the linkage between financial resources and a quality faculty 
body. The factor loadings show that the previous notion of quality described earlier 
can be better defined as undergraduate quality, as the six -year graduation rate, 
freshmen retention rate, percent of undergraduates living in dorms, and the freshmen 
acceptance rate are particularly relevant to the undergraduate experience. The 
percentage of both degrees granted and enrollment that is graduate form a distinct 
factor of their own, explaining 4.1 percent of the total variance. It is somewhat 
surprising that the student faculty ratio does not load heavily with any other measures, 
but by itself, it does explain 3.6 percent of the total variance. Finally, tuition and fee 
revenue and state appropriations per student form another financial factor that can be 
characterized as financial support, and distinct from the previous financial factor that 
is dominated by expenditure measures. Interestingly, state appropriations per student 
has a negative loading while tuition and fee revenue per student loads positively. This 
is perhaps indicative of recent experiences in public higher education whereby tuition 
and fee revenues are increased to offset decreased or stagnant state funding. 

Appendix C illustrates the dendogram that graphically depicts the institutional 
clustering sequence that results when the factor scores are submitted to the clustering 
algorithm. Of primary interest in this exercise are those institutions that cluster with ■ 
Albany, the reference institution. Examining the dendogram shows that Albany first 
clusters with SUNY Binghamton. These two institutions are then joined by William 
and Mary. This small cluster of three institutions is then joined by a larger cluster 
composed of SUNY Buffalo, U South Carolina, U Connecticut, SUNY Stony Brook, 
U Virginia, and UNC Chapel Hill. Finally, Georgia Tech, which has not joined a 
cluster by this stage, joins the Albany cluster. Institutional clustering can continue all 
the way to the point when all of the institutions are amalgamated into one large 
cluster. 



For the purposes of this research, cluster formation was stopped at this point, 
when overall, eight distinct clusters are formed. The nine most similar institutions to 
Albany noted above are identified as belonging to its peer cluster. Contunued 
clustering would eventually combine a cluster of eight additional institutions (U 
Wash., U Minn., U Kentucky, U Cinnci., U Alabama Birm., U Pitt., U Mich. Ann- 
Arbor, and U Vermont) to the Albany cluster. Other clusters would also combine 
though, resulting in a four cluster solution. UC San Diego and U Hawaii-Manoa, 
however, still fail to cluster with other institutions at this point and continue to 
comprise single institution clusters. While Albany’s peer cluster would contain 
eighteen institutions, the other large cluster would contain sixty-nine institutions. It is 
felt that inclusion of the additional eight institutions into the peer cluster can not be 
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justified on the grounds that doing so would unduly reduce homogeneity between 
clusters, as evidenced by the formation of the extraordinarily large cluster. 

What can be said of the peer cluster that contains Albany and nine other 
institutions though? Five of the ten have medical schools, Albany does not. One land 
grant institution, U Connecticut, is included, although one does not usually think of U 
Connecticut as one of the traditional large mid-western land grant universities. Six of 
the ten institutions are classified as Carnegie Research I, two, including Albany, are 
Research II, and two are Doctoral I. Table 3 displays these mission characteristics for 
the peer cluster, as well as for those institutions identified in the rank distance 
analysis as peers, but which failed to cluster with Albany. The percentile rank of each 
institution on the seven factor dimensions are also shown. 



Table 3 



Distanc Misaion 



Analysis 


Medical 


Land- 




ffromAIbanvl Cluster 


School 


grant 


Carnegie 


SUNY Albany 


— w/Albany 






R2 


SUNY Binghamton 


1.47 w/Albany 






D1 


William & Mary 


1.76 w/Albany 






Dl 


SUNY Buffalo 


1.65 w/Albany 


Yes 




R1 


U So. Carolina 


1.70 w/Albany 


Yes 




R2 


U Conn 


1.58 w/Albany 




Yes 


Rl 


SUNY Stony Brook 


1.89 w/Albany 


Yes 




Rl 


U Virginia 


1.91 w/Albany 


Yes 




Rl 


UNC Chape! Hill 


2.23 w/Albany 


Yes 




Rl 


Georgia Tech 


1.68 w/Albany 






Rl 


U Oregon 


1.64 3 






R2 


Delaware 


1.70 2 




Yes 


R2 


UC Santa Barbara 


1.70 2 






Rl 


Virginia Polytech 


1.73 1 




Yes 


Rl 


UC Santa Cruz 


1.74 2 






R2 


Clemson U 


1.79 I 




Yes 


R2 


Virginia Commonw 


1.79 3 


Yes 




Rl 


U Rhode Island 


1.79 2 






R2 


Oregon State U 


I.8I 1 




Yes 


Rl 


U Mass-Amherst 


1.82 2 






Rl 


Washington State 


1.83 1 


Yes 


Yes 


R2 



Factor Score Percentile Rank 

General Stu/Fac Financial 



Size Financial 


Quality Complexity 


Graduate 


Ratio 


Support 


22.5 


27.0 


91.0 


78.7 


93.3 


85.4 


53.9 


6.7 


11.2 


98.9 


88 8 


77.5 


65.2 


12.4 


2.3 


15.7 


100.0 


34.8 


89.9 


30.3 


52.8 


67.4 


40.5 


77.5 


83.2 


95.5 


56.2 


29.2 


62.9 


13.5 


52.8 


42.7 


97.8 


44.9 


60.7 


52.8 


19.1 


85.4 


75.3 


85.4 


21.4 


84.3 


25.8 


83.2 


70.8 


89.9 


96.6 


43.8 


50.6 


47.2 


75.3 


97.8 


47.2 


98.9 


53.9 


86.5 


61.8 


93.3 


96.6 


58.4 


80.9 


10. 1 


3.4 


3.4 


97.8 


95.5 


1.1 


94.4 


74.2 


27.0 


29.2 


34.8 


39.3 


67.4 


56.2 


94.4 


80.9 


36.0 


36.0 


83.2 


73.0 


28.1 


42.7 


96.6 


39.3 


46.1 


73.0 


94.4 


3.4 


97.8 


66.3 


70.8 


44.9 


88.8 


9.0 


64.0 


28.1 


59.6 


9.0 


52.8 


60.7 


96.6 


l.I 


66.3 


78.7 


19.1 


55.1 


86.5 


13.5 


53.9 


47.2 


30.3 


24.7 


62.9 


14.6 


61.8 


87.6 


48.3 


75.3 


18.0 


38.2 


48.3 


77.5 


40.5 


34.8 


95.5 


20.2 


89.9 


37.1 


19.1 


32.6 


69.7 


41.6 


69.7 


23.6 


74.2 


80.9 


24.7 


33.7 


93.3 


30.3 


76.4 


68.5 


14.6 


11.2 


32.6 


36.0 



As Table 3 illustrates, each institution in the peer cluster, except for UNC 
Chapel Hill, with a distance analysis score of 2.23 (34 positions away from Albany), 
were held in the rank distance analysis as potentially peer eligible. The more ethereal 
factor dimensions, while not as interpretable as the individual measures that compose 
them, can nonetheless still be evaluated in respect to the percentile rank of individual 
institutions. The institutions in the peer cluster rank similarly high with respect to 
quality and graduate education emphasis. Overall, these institutions appear to rank 
nearer to the median on the student faculty ratio dimension, with Albany and Georgia 
Tech having the highest percentile ranks. The size, general financial, and financial 
support dimensions show more variability among institutions in their respective 
rankings. The complexity dimension, while not as diverse, shows a skewed 
distribution, with Georgia Tech, noted for its engineering emphasis, having a 
percentile rank of only 1.1. 
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Discussion 

In summary, the rank distance and factor and cluster analysis techniques 
provided an overlapping group of peer institutions for the reference institution, SUNY 
Albany. In both analyses, campus decision makers played an integral role in 
developing the list of variables and their respective measures that were used to 
differentiate institutions. With both techniques, the institutional research staff, as 
well as higher level campus decision makers, can clearly see how Albany compares to 
other institutions across financial, size, administrative complexity, graduate study 
emphasis and quality measures. Increased familiarity with and Iqiowledge about these 
aspects of our own institution and of our chosen peers is of crucial importance to 
SUNY Albany in a constrained resource environment. 

There are though limitations to these methods. The rank distance method 
employed did not assign weights to the individual measures. While an implicit 
weighting scheme was in effect due to the different number of measures used to 
represent finance, size, quality, and complexity variables, arguments can certainly be 
made to alter the variable weighting. At Albany, a decision was made to seek peer 
institutions more closely related on the financial and size variables, and more of those 
measures were used, precluding the need for weights. 

Another limitation of the rank distance method is that basing the distance 
measure on percentile ranks instead of on factor or standardized scores distorts the 
magnitudes of the differences between institutions. The percentile rank approach 
creates a distance measure based on the rank order of institutions for different 
measures rather than upon the magnitude of their differences from each other. The 
actual distances in the raw data may be greater or smaller than the distance between 
ranks. 



A final limitation of this methodology worth noting is that many of the 
measures used, as noted, are highly correlated with each other. In other words, to a 
certain degree, they may be measuring the same concepts. While the measures used 
address different concerns campus decision makers harbored, the end result may be 
confounded by the highly related nature of the measures. 

The factor and clustering technique, which alleviates the problem of highly 
correlated measures by factoring them into seven completely uncorrelated 
dimensions has an advantage in this respect. In this analysis each of the seven 
dimensions were equally weighted. A weighting scheme could though be introduced 
to stress the importance of some dimensions over others if desired. 

A well noted limitation of factor analysis is that the factors (dimensions) used 
for clustering are more difficult to assess. This is particularly true with respect to the 
percentile ranks used in the first analysis. While standardized factor scores do not 
easily lend themselves to meaningful interpretation, an institution’s percentile rank on 
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each factor score can be computed to provide an indication of where an institution 
ranks on particular dimensions, vis a vis other institutions. 

Both of these methods provide a means of identifying peer institutions. 

Neither should be viewed as a turn-key approach. The rationale for the undertaking in 
the first place plays a significant role in choosing variables and measures of them for 
consideration. Acceptance of a peer institution group by campus decision makers and 
relevant external audiences largely depends on accommodating and incorporating 
their intuition, concerns, and political objectives. The peer review process is a 
learning process. Through it we learn not only about the funding, quality, or size of 
other institutions, but more importantly, we learn about how our own institution 
stands in those respects. 
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WEAVING INSTITUTIONAL RESEARCH INTO THE FABRIC OF TQM 

Dawn Geronimo Terkla 
Executive Director of Institutional Research 
Tufts University 

Kelli J. Armstrong 
Research Analyst 
Tufts University 



Introduction 

As academic institutions continue to look to the business community for 
models of efficient practice, many universities have adopted the theories behind Total 
Quality Management (TQM) for both academic and administrative improvements 
(Manley, 1996). Because the process of TQM necessitates data gathering -- a point 
where a quality improvement team must examine prior practices and assess current 
status — the expertise of an office of institutional research staff is often utilized. 

The needs of quality improvement (QI) teams and the practices of an 
Institutional Research Office may find themselves at odds, however. Data gathering 
for research projects at most IR offices necessitates systematic planning and careful 
analysis, yet the time demands of a QI team often do not allow for weeks of research. 
This paper will examine the impact of improvement techniques like TQM on the 
resources of an institutional research staff and the approach that has been adopted at a 
Research I University to reach a compromise between the data needs of QI teams and 
good research practice. 

TQM Methods for Higher Education 

In the past decade quality improvement techniques have been widely 
documented in the field of higher education. Institutions use the techniques to 
improve specific areas, such as classroom teaching practices and administrative 
processes, or the quality improvement approaches may be wide-reaching and 
incorporated as part of an overall assessment process (Sherr and Teeter, 1991, 
Palmer, 1996). 

Although many TQM resource guides provide useful information on how to 
diagram process problems, translate TQM findings into practice, facilitate TQM 
groups, or sell the concept of quality improvement to the institution at large, few 
guides provide details or "maps" on how to gather data to inform the improvement 
process (Teeter & Lozier, 1993; Ruben, 1995). The reason for the absence of 
specific material is understandable: each TQM problem (and thus the data needed to 
address the problem) is unique and therefore requires a distinct approach to the 
research design and subsequent data collection. 

Without some kind of prescription for data collection, however, QI teams 
may be hindered in their work by this stage of the process. Many teams lack 
adequate tools to gather the information they need to answer a research question. 
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Simply enlisting the assistance of an Institutional Research Office can be problematic, 
though. Most trained researchers have well-defined approaches to the process of 
gathering and analyzing data. In the carefully dictated time limits for a TQM team to 
produce a resolution, data gathering must be quick and efficient, with timelines often 
not conforming to the normal standards used in social science research practice. The 
following outlines how one institution, through trial and error, worked towards a 
research compromise for a newly created TQM program on campus. 

The Adoption of TQM and the Effect on an IR Office 

At a mid-sized research university, Total Quality Management principles were 
adopted by the senior level administration and techniques were modified to meet 
specific university goals with the expectation that these tools could be used by all 
facets of the institution. A TQM steering committee was established to oversee the 
university-wide TQM effort and to establish the priority by which projects were 
addressed. The central coordination function for TQM projects was assigned to the 
Office of Development and Training (ODT). The ODT staff developed the campus- 
wide TQM protocol for the Quality Improvement teams, involving seven basic 
process steps. As one of the stages in the standard TQM process required a research 
component, ODT enlisted the assistance of the Office of Institutional Research for 
support and to serve as an in-house consultant. 

In less than two years, close to twenty Quality Improvement (QI) teams were 
established to examine a variety of issues including such projects as the scheduling of 
the budget process, access of orientation information for new employees, and a re- 
examination of the entire process of pre-health advising for undergraduate students. 
With the exception of a few early teams, the Office of Institutional Research 
consulted or managed the research component for all QI teams on campus. 

Although Institutional Research staff members had experienced on-campus 
training in the philosophy and team-based techniques of TQM, the process of 
managing the research component for QI teams posed a new challenge. Each team 
had unique needs and a new cast of characters. Overall, the impact on the IR staff 
was enormous; in a period of less than a year, the project list for the office more than 
doubled. Typically each team would require the assistance of two members of the IR 
staff who needed to be present at all relevant meetings and involved in various aspects 
of survey instrument design, data collection, analysis and presentation. 

TQM Teams and IR Difficulties 

In addition to managing the volume of new projects, the IR office needed to 
develop strategies to effectively provide services to QI teams. During the initial 
period, sometimes it was difficult to easily determine services that were required. 

Like all new practices, unexpected problems often arose which slowed or hampered a 
team's progress. 

As each newly-formed QI team examined widely different problem statements 
and outcome goals, the research needs and data requirements of the groups varied 
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accordingly. In addition, the research background and training of team members were 
uneven, ranging from those who had little or no experience with data analysis to 
those who had extensive training in social science or natural science research. The IR 
office also was placed in the precarious situation of maintaining an advice-giving 
position, yet not possessing actual membership on the QI teams. When offering 
research advice and support, it was important not to prescribe the type or form that 
the research component should take in the TQM process, but rather present alternative 
strategies and highlight their strengths and weaknesses. 

In the course of the first year working with QI teams, the IR office 
encountered a number of these tangles which fell into several broad categories. 

1- Lack of research training/understanding: QI teams were comprised of 
individuals from a wide variety of backgrounds according to their relationship with 
the issue at hand. Some teams were fortunate to have members with strong social 
science or natural science research training, or business marketing training. However, 
there were other teams that did not have this advantage as their team members had 
little to no background in the process of gathering and analyzing data. 

The university's TQM framework allows 3 to 4 weeks for initial data 
collection to inform the quality improvement process. Team members with little data 
collection experience needed to quickly become familiar with a few of the basic 
research techniques in order to move their QI team to the next stage. Once the data 
were collected, they also needed additional quick tutoring to assist them in the 
interpretation of data. For many of the QI teams, developing instruments to gather 
the data needed to answer research questions proved to be one of the most difficult 
aspects of the TQM process. Thus, the IR staff was faced with the challenge of 
providing a broad overview of research methods and analysis techniques. 

2- Scope of Problem Not Manageable: On several occasions the IR staff 
encountered a team who had designed a research problem that was much too large to 
analyze in its entirety during such a short data collection period. The research 
required to effectively examine these "large" issues would greatly exceed the typical 
TQM timeline. Thus, the IR staff needed to help re-focus and narrow the problem to 

a more measurable research statement. . 

3- Too many data cooks: Many QI teams determined that the most 
appropriate method to gather information was through a survey instrument. Several 
'.teams attempted to be all-inclusive and circulated drafts of the instruments to each 
iteam member, at each draft stage. With the Institutional Research staff coordinating 
the development of the instrument, on occasion an instrument would be altered a 
cdozen times within a one-week period to respond to the comments of all team 
members. Given the strict timeline for data collection, this painstaking process 
iproved to be a roadblock towards maintaining the team's schedule. Thus the IR staff 
was faced with the challenge of making the survey development process inclusive, but 
not cumbersome. 

4- Political Pitfalls: Often when IR staff members arrived on the scene to 
provide support for the research collection stage, they would be unaware of the 
political undercurrents existing within each individual QI team. Although a particular 
method of data collection might appear at first glance to be the most efficient, several 
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team members might oppose the method due to pre-existing political climates. Issues 
such as confidentiality and the idiosyncracies of inter-departmental cultures can create 
barriers regarding the types of questions to be asked and how responses should be 
collected. Thus the IR staff was faced with how to avoid bending to a particular 
team's political necessities which could result in the collection of data that could 
leave many of the research questions still unanswered. 

5- Too much data! In addition to the management of instrument design and 
data collection for the TQM teams, the IR office was often asked to analyze and 
present the data in a useful form. In an effort to be as helpful as possible, the IR 
office initially attempted to anticipate every possible question the team might have 
from the data. Early data reports were quite voluminous, including a multitude of 
tables and graphs with the hope that if a TQM team had a question about the data, 
they would be able to find the answer in the report. Unfortunately, many of the teams 
were overwhelmed by the sheer size of the data report and found that the volume of 
information actually impeded their process. Thus, the IR staff was faced with the 
challenge of presenting data in a more "user friendly" form. 

Using TQM to Improve TQM Research 

In order to accomodate and plan for the burgeoning needs of this new campus- 
wide endeavor, the IR staff worked with the ODT staff to continually improve their 
communication and interactions. Regular meeting times between the two staffs were 
established so that ongoing feedback on successful and less-than-successfiil practices 
could be discussed. 

One of the policies which emerged from these meetings was that the ODT 
staff agreed to include Institutional Research staff members earlier in the TQM 
process, keeping them informed as soon as a TQM proposal was received so that they 
could plan their resources accordingly. The earlier involvement helped avoid some 
of the inner QI team political difficulties the IR staff had encountered when it arrived 
late in the process. 

To address the problems of teams requesting too much data analysis or 
addressing research statements too large for the timeframe, the practice was 
established of having IR staff members meet with the QI team leader and facilitator 
prior to the research process to further clarify team needs. The team leader and 
facilitator also helped establish a QI subcommittee to work on the detailed parts of 
the data collection and analysis, thereby avoiding the "too many data cooks" problem 
encountered earlier. 

To address the problem of varied research backgrounds of team members, the 
IR office began its consulting role by composing a "Data Collection Primer" to be 
included in the TQM manual. In this primer, basic research vocabulary were defined 
with examples, such as how to develop a research question, the difference between 
quantitative and qualitative research, and ways to analyze data in order to provide a 
basic level of understanding about the research process for all TQM team members. 



'Copies of the "Data Collection Primer” are available upon request. 
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Members of the IR staff met with the TQM teams to present the primer and to answer 
early research-related questions. Over time, the Data Collection Primer was 
modified to become more of a "workbook" so that QI teams would approach the data 
process better prepared to discuss specific research problem areas. 

The Institutional Research Office also changed its usual approach to the 
presentation of data reports. The data presentations which were most useful to TQM 
teams were quite different than those completed by the IR office for earlier 
institutional projects. Generally, the data presentations needed to be simpler, limited 
only to results relevant to the research problem and presented graphically. Team 
leaders were also given the option of requesting additional information reports 
derived from the data, as needed. 



Conclusion 

In many ways, the interaction of the Institutional Research Office with the 
TQM endeavor became its own quality improvement process. Over the course of a 
calendar year, the IR staff continually searched for new ways to serve the various 
TQM teams on campus, moving from its standard practice of conducting research 
endeavors using longer-range social science techniques, to an increasingly targeted, 
market-based approach. The close timeline would often intensify the process; 
instead of mailing follow-up surveys to increase response rates, for example, 
telephone calls to survey recipients would be made by team members and IR staff. 

The basic tenets of TQM — continous improvement of services to customers and 
greater efficiency — were used to help the Institutional Research Office provide better 
data gathering tools for the TQM team members. 
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IR INFLUENCE ON MARKETING AND PRICING POLICIES: 
FIRST-TIME GRADUATE STUDENT INQUIRIES-WHY DON’T THEY APPLY? 

Stephen W. Thorpe 
Director of Institutional Research 
La Salle University 



With increased competition for full-time undergraduate students, institutions 
of higher education have developed part-time adult undergraduate and graduate 
programs of study. For one urban, private university, part-time graduate programs 
were developed with the intention of attracting new markets of adult learners. 

Each year. La Salle University receives thousands of inquiries for their part- 
time graduate programs, and yet very few ultimately apply for admission. The 
graduate program directors believed that tuition costs were the primary deterrent to 
increasing applications for admission. The argument was that graduate programs 
could enroll more students if only the university would lower tuition costs, either by 
lowering the sticker price or by lowering the net price with financial aid. Other 
administrators, however, questioned the impact of price on applications compared to 
non-price issues, including program offerings, convenience, and location. These 
discussions, which were ongoing for several months, did not seem to be leading to 
new strategies for graduate student recruitment or program development. 

The Director of Institutional Research was invited to assist in efforts to inform 
decision making regarding marketing and pricing issues for graduate programs. In 
this role, the Director served as an internal market research consultant to both senior 
administrators and the graduate directors. 

Through initial discussions with administrators, several factors were identified 
as impacting recruitment of adult graduate students: the institution was not 

effectively marketing its programs to targeted audiences; tuition prices were higher 
than the local competition; potential students were not typically eligible for employer 
reimbursement of tuition; the existing programs might not be aligned with the 
interests of the public; and the location of the institution was inconvenient to work 
sites or homes. Upon further discussion, however, it became clear that the 
demographics of the potential graduate students were unknown, thus making 
conclusions about their ability to pay, interests in programs, and employment status 
unclear. Moreover, identification of the competition was problematic since the 
graduate programs were either general (and therefore several competitors existed in 
the irrunediate area) or they were unique to the area and served very specific 
audiences. 

As a result of these initial conversations, the Director of Institutional Research 
recorrunended that a survey of inquirers to the graduate programs be conducted. 
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Through further consultation with the graduate directors, a Graduate Inquiry Survey 
was developed.' The survey was designed to measure the potential factors identified 
by the graduate directors as issues affecting recruitment. In addition, the survey 
instrument sought to identify the competitor schools for each graduate program. 

The Graduate Inquiry Survey was mailed to inquirers of eight of the graduate 
programs at La Salle: Bilingual Studies, Professional Communication, Computer 

Information Science; Education; Nursing; Pastoral Counseling; Psychology; and 
Religion. The sample included individuals from Delaware, Pennsylveinia, eind New 
Jersey who inquired during the previous year but did not apply for admission. 



Table 1 

Sample Sizes and Response Rates 
by Graduate Program 



Graduate Program 


Sarnple Size 


Response 


Response Rate 


Bilingual Studies 


179 


33 


18.4% 


Communications 


190 


39 


20.5% 


Comp Info Science 


190 


29 


15.2% 


Education 


182 


32 


17.6% 


Nursing 


162 


34 


21.0% 


Pastoral Counseling 


167 


56 


33.5% 


Psychology 


183 


40 


21.9% 


Religion 


129 


28 


21.7% 


Totals 


1,382 


291 


21.0% 



The sample consisted of up to 200 inquirers randomly selected for each 
graduate program. All inquiries were included in the sample for those programs that 
received fewer than 200 inquiries during the period of study. The selection of 200 
inquiries per program was based on budgetary considerations. Two weeks after the 
initial survey was mailed, a follow-up sm^^ey was sent to those inquirers who had not 
yet responded. 

Table 1 shows the response rates by graduate program. From a sample of 
1,469 inquiries, 87 surveys were returned as undeliverable. Of the remaining 1,382 
individuals, 291 returned a survey, for a response rate of 21%. Although the response 
rates varied by program, the overall response rate was considered impressive since the 
respondents had no allegiance to La Salle. 



’ A copy of the survey instrument is available from the author. 
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Survey Results 



One of the important outcomes of the study was to describe the potential 
graduate student market that was inquiring at La Salle. Table 2 presents several of the 
results. Nearly 75% of the graduate inquirers said they employed full-time; however, 
less than half of the inquirers said they were eligible to receive tuition reimbursement 
for graduate study from their employers. 



Table 2 



Employment Status and Plans for Graduate Study 





N 


% 


Emnloved Full-time? 


Yes 


212 


75% 


No 


72 


25% 


Eligible for emnlover reimbursement? 


Yes 


108 


48% 


No 


98 


44% 


Not Sure 


18 


8% 


Plans for Graduate School? 


Currently applying at La Salle 


5 


2% 


Applied/Enrolled Elsewhere 


89 


39% 


Plaii to Apply at LSU later 


76 


33% 


Decided against graduate school 


60 


26% 



About 40% of the graduate inquirers said they applied or enrolled at another 
institution after receiving information about La Salle’s graduate programs. 
Surprisingly, another 26% said they decided against graduate school for the near 
future. These results suggested that the inquiry pool was somewhat “soft” inasmuch 
as one of every four inquirers ultimately decided not to attend graduate school. 

The principal purpose for the study was to determine why individuals do not 
apply for admission after inquiring about La Salle’s graduate programs. The 
respondents were asked to select from a provided list those reasons that were 
applicable to their decision not to apply for admission. In addition, the respondents 
were asked to identify the one reason that was most influential in their decision not to 
apply. As Table 3 shows, the reasons most often cited for not applying for graduate 
admission were “tuition too expensive: (29%), “not clear on career goals” (26%), 
“location” (24%), “job commitments”(22%), and “lack of financial aid” (21%). The 
reasons identified as most important included “tuition too expensive” (18%), “job 
commitments” (17%), “program did not meet needs” (11%), and “location” (10%). 
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These results suggested that while La Salle was losing a portion of the inquiry pool 
because of cost, just as many inquirers were not applying for admission because of 
external forces, such as job commitments and unclear career goals of the inquirers. 



Table 3 



Reasons for Not Applying for Admission 





N 


% 


Reasons for Not Annlvine 


Tuition too expensive 


83 


.29% 


Not clear on career goals 


74 


26% 


Location 


67 


24% 


Job commitments 


61 


22% 


Lack of financial aid 


59 


21% 


Family responsibilities 


49 


17% 


Program did not meet needs 


43 


15% 


Prog/specialization not available 


39 ^ 


14% 


Total Responses 


283 


100% 


Most ImiDortant Reasons 


Tuition too expensive 


48 


18% 


Not clear on career goals 


46 


17% 


Program did not meet needs 


31 


11% 


Location 


26 


10% 


Total Responses 


270 


100% 



In addition to examining the reasons for not applying across all inquirers. 
Table 4 presents the reasons for not applying for just those inquirers who said they 
had applied or enrolled elsewhere. Examining this sub-group was especially helpful 
since these individuals, who represented nearly 40% of the respondents, actually 
applied or enrolled at another institution which demonstrated their commitment to 
graduate study. 

The reasons most cited for not applying to La Salle for this sub-group included 
“tuition too expensive” (40%), “location” (34%), and “program did not meet needs” 
(31%). While tuition expense was cited most often by the overall sample, it was 
interesting to note that almost as many inquirers in the sub-group cited location and 
programs not meeting their needs as reasons for not applying to La Salle. In addition, 
the most important reasons for not applying were equally “tuition too expensive” 
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(25%) and “program did not meet needs” (22%). To a lesser extent, “location” (12%) 
was also identified. These results suggested that La Salle was losing potential 
students because of both cost and program offerings. 



Table 4 



Reasons for Not Applying for Admission 
(Inquiries for Applied/Enrolled Elsewhere) 





N 


% 


Reasons for Not Annlvine 


Tuition too expensive 


34 


40% 


Location 


29 


34% 


Program did not meet needs 


27 


31% 


Prog/specialization not available 


19 


22% 


Total Responses 


86 


100% 


Most Imnortant Reasons 


Tuition too expensive 


21 


25% 


Program did not meet needs 


19 


22% 


Location 


10 


12% 


Total Responses 


85 


100% 



For those inquirers who applied or enrolled at another institution, the Graduate 
Inquiry Survey asked them to identify those features of the graduate program they 
selected that were more attractive compared to La Salle’s graduate program. As 
Table 5 shows, the primary features identified as attractive about the competition 
were “program met needs” (62%), “course offerings” (47%), “more appropriate to 
career goals” (42%), “convenient location” (40%), and “lower cost” (39%). The 
primary features of the competitor programs identified as most attractive were 
“program met needs” (29%), “lower cost” (13%), “more appropriate to career goals” 
(12%), and “convenient location” (8%). These results suggested that tuition pricing 
alone was not likely to dramatically affect graduate enrollment. Rather, issues of 
program offerings that are targeted to potential career goals and convenient locations 
for graduate programs must also be addressed. 

In addition to presenting the overall findings of this study to the graduate 
directors, survey responses were examined by graduate program. These results, not 
shown in this paper, revealed that the graduate programs attracted student markets 
that had different needs and priorities. Moreover, issues such as cost or location 
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affected these groups of potential students differently, which suggested further 
implications for marketing and pricing. 



Table 5 

Attractive Features of Competitor Schools 
(Inquiries for Applied/Enrolled Elsewhere) 





N 


% 


Attractive Features 


Program met needs 


53 


62% 


Course offerings 


40 


47% 


More appropriate to career goals 


36 


42% 


Convenient location 


34 


40% 


Lower cost 


33 


39% 


Total Responses 


85 


100% 


Most Attractive Feature 


Program met needs 


24 


29% 


Lower Cost 


11 


13% 


More appropriate to career goals 


10 


12% 


Convenient Location 


7 


8% 


Total Responses 


83 


100% 



In summary, the Survey of Graduate Inquiries provided several findings that 
directly affect marketing and pricing decisions for part-time graduate programs for 
this institution; (1) the apparent differences in potential student markets for the 
graduate programs should be considered in all pricing and marketing discussions; (2) 
the inquiry pool is relatively “soft”; and (3) while tuition cost was a deterrent for 
some programs, other issues such as program offerings that meet the needs of 
potential students and the location (or even convenience) of the university were also 
very real deterrents for inquirers who did not apply for admission. In addition, this 
research showed that tuition cost was less of an issue for those inquirers who enrolled 
at other institutions, again implying that program offerings were deterring potential 
students more so than cost. 

Conclusion 

This research project demonstrated the role that institutional researchers can 
play as internal consultants. In this case, the Office of Institutional Research was 
pivotal in not only conducting the market research study, but also in informing future 
policy discussions regarding the factors that impact on the recruitment of adult 
students for part-time graduate programs. 
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INTRODUCTION 

Students, faculty, and administrators have long been interested in measuring 
aind understainding the relative reputations of universities and individual programs 
within these universities. To address this issue, academics have attempted to rate and 
rank programs particularly at the graduate level (e.g., Hughes 1925, 1934; Keniston 
1959; Cartter 1966; Roose and Andersen 1970; Jones, Lindzey, and Coggeshall 1982; 
Goldberger, Maher, and Flattatu 1995). The results from these studies have often sent 
administrators aind faculty members searching for what the ratings say about the 
quality of their programs and what can be done to improve them. 

Despite the wealth of surveys aimed at measuring program quality, much 
debate exists within academe as to the validity of these ratings, and even what 
concepts of “quality” are being measured. To date, most attention (and statistical 
analyses) has focused on the scholarly quality of faculty as the best measure of 
reputation. These studies have been criticized by some because they did not address 
the teaching mission of research universities. The scholarly quality ratings generated 
by peer assessments have been further criticized on the grounds that they are 
influenced by "rater bias" (Tan 1986). Such distortions in the ratings may occur if, for 
example, peers rate programs partially on the basis of where the program is located in 
addition to what the faculty have accomplished in research and teaching. If ratings 
are influenced by "non-quality related" factors, then this will limit the extent to which 
the ratings are a useful guide to participants in the higher education system. 

Dating back to the survey by the American Council of Education (ACE), most 
studies have asked raters to provide separate evaluations of programs based on the 
"quality of the graduate faculty" and the "effectiveness of the doctoral program". 
However, it is important to ask whether raters have sufficient information on both the 
(research) quality of the faculty and the (teaching) effectiveness of the program to 
make good judgements about each for a large number of programs. For example, in 
the 1993 survey conducted by the National Research Council (NRC), raters were 
instructed to consider "the scholarly competence and achievements of the faculty" for 
measuring scholarly quality (Goldberger et al.; 1995, p.l24). However, the list of 
criteria to be used for evaluating program effectiveness included: 

"The accessibility of the faculty, the curricula, the instructional and research 
facilities, the quality of graduate students, the performance of graduates, the 
clarity of stated program objectives and expectations, the appropriateness of 
program requirements and timetables, the adequacy of graduate advising and 
mentorship, the commitment of the program in assuring access and promoting 



success of students historically underrepresented in graduate education, the 
quality of associated personnel (post-doctorates, research scientists, et. al.), 
and other factors that contribute to the effectiveness of the research-doctorate 
program" (1995, p.l24). 

Clearly scholarly quality would seem to be easier for raters to evaluate since 
the research output of faculty is more readily available to individuals outside of the 
program than the listed aspects on teaching effectiveness. If raters have less 
knowledge about the teaching effectiveness of individual programs than they do about 
the scholarly quality of faculty in programs, then raters may simply use their 
impressions of scholarly quality to make inferences about the program effectiveness 
in instruction. 

This paper summarizes some of our key findings with regard to the factors that 
influence the scholarly quality and program effectiveness ratings of graduate 
programs in the 1993 survey sponsored by the National Research Council. A paper 
containing the complete statistical results from our analysis, including a more 
thorough discussion of the literature and a comparison of alternative regression 
models by major groupings of fields, is available upon request. We use multiple 
regression analysis to examine how previously identified faculty, student, and 
institutional characteristics influence the recent NRC program ratings of both 
scholarly quality and program effectiveness, and whether other factors, such as the 
reputation of the institution and whether the institution is public or private, also affect 
programmatic ratings. 

Our results show that not only are both quality measures very highly correlated 
with each other, but they are influenced in similar ways by factors such as research 
output, departmental size, and institutional characteristics. While we found that 
measures of research output and the size of the program were very important 
determinants of program ratings, the data also show that the institutional 
characteristics of where a program is located are also important factors in the 
program’s ratings. Peers tended to rate programs in private institutions higher than 
those in public institutions, and also rated programs in institutions with many other 
highly-rated programs higher than similar programs in other institutions. 

THE LITERATURE 

Several literature reviews exist on the studies that have been conducted 
pertaining to various measure of academic quality and programmatic ratings (see Tan 
1986; Conrad and Blackburn 1986). Early pioneers in the study of academic program 
ratings include Hughes (1925, 1934) and Keniston (1959), and perhaps the best- 
known early studies of program ratings were conducted by the American Council on 
Education, as reported by Cartter (1966) and Roose and Andersen (1970). The 
National Research Council performed their first study of program reputations in 1982, 
and in the fall of 1995 released the results from a similar study conducted in 1993. 

The 1 993 NRC study is notable for the large number of faculty used as raters (over 
16,700), and have become widely used both within academia and in the popular press 
as a barometer of the reputation of graduate programs. 
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There is still considerable debate within academe as to what "quality" is and 
how it should be measured. Following the reputational studies, a number of 
researchers have searched for factors that may help to explain the quality ratings of 
doctoral programs (e.g., Hagstrom 1971; Saunier 1985; Conrad and Blackburn 1985; 
King and Wolfle 1987; Fairweather 1988; Tan 1992; Ehrenberg and Hurst 1996a, 
1996b). These studies varied in the statistical and empirical methodologies 
employed, the objective measures used to explain ratings, and the survey data 
analyzed. Since we use multiple regression analysis in this study, it is useful to 
briefly highlight other studies that have used regression analysis. Hagstrom (1971), 
using the ACE ratings of programs in four fields, grouped regressors into six 
categories and estimated separate regression models for each category of regressors. 

A similar procedure was used by Conrad and Blackburn (1985) to examine the ratings 
derived from separate program reviews by several teams of external reviewers. By 
omitting other key independent variables from each regression model, however, these 
studies cannot identify the partial effects of variables of interest holding other 
important factors constant. 

Morgan, Kearney, and Regens (1976) also used multiple regression analysis to 
examine the ACE ratings for graduate programs in the humanities, social sciences, 
natural sciences, and humanities. More recently, Fairweather (1988) applied multiple 
regression analysis to explain the program ratings from the 1982 NRC study for three 
fields. Saunier (1985) applied stepwise regression to the program ratings in six fields 
obtained from the 1982 NRC study. Finally, Ehrenberg and Hurst (1996a, 1996b) 
used multiple regression analysis to examine the program ratings from the 1993 NRC 
study. They found that there was a quadratic relationship between faculty size and 
scholarly quality ratings, and a negative relationship between the dispersion of 
publications (as measured by the Gini coefficient) and the scholarly quality ratings. 
However, Ehrenberg and Hurst did not examine whether the scholarly quality ratings 
were influenced by institutional characteristics. All three of these studies estimated 
models separately for each field, and focused exclusively on explaining the scholarly 
quality of faculty. 

Taking the correlate and multivariate studies together, over the past three 
decades studies consistently demonstrated that research output has been one of the 
most significant variable associated with the reputational ratings of university 
doctoral programs. They concluded that this is probably due to the fact that they are 
the primary means of gaining scholarly recognition and reputation. . Another 
consistent finding was that the size of a program was a significant contributor to the 
reputation of the program. 

DATA DESCRIPTION 

In this study, we examine the factors influencing programmatic ratings for 
thirty fields spanning four groupings of fields: (i) biological sciences, (ii) physical 
sciences and mathematics, (iii) social and behavioral sciences, and (iv) engineering. 
After eliminating missing values, there were a total of 1,809 programs that we used in 
the final analysis. We excluded programs from the arts and humanities in this study 
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because the NRC dataset did not contain information on publications and citations for 
these fields. 

Our analysis is distinctive from other multivariate studies in several respects. 
First, rather than restrict our analysis to a particular field, we estimate multiple 
regression models over the pooled sample of fields. The pooling of the data allows us 
to make more precise estimates of the coefficients for factors in the model than would 
be possible when estimating models for each separate field. Second, rather than focus 
exclusively on the scholarly reputation of the faculty rating for each program, we 
consider two alternative programmatic rating variables, each based on responses from 
a national sample of faculty in each of the relevant disciplines, in the models that 
follow. The first variable is the mean scholarly quality rating of the program's faculty, 
which represents "the" scholarly competence and achievements of the faculty" rated on 
a six-point scale ranging from 0="not sufficient for doctoral study" to 
5="distinguished". The second variable is the program effectiveness, which 
represents "the program effectiveness in educating research scholars and scientists" 
rated on a six-point scale ranging from 0="not effective" to 5="extremely effective". 

EMPIRICAL RESULTS 

We began our analysis by using the variables made available by the NRC that 
have been used by previous researchers to explain scholarly quality ratings. The 
variables include measures of program characteristics (e.g., publications per faculty), 
student characteristics (e.g., percentage of graduate students who are teaching 
assistants), and institutional characteristics (e.g., total library holdings). We also used 
the concentration of publications or citations within a program, as measured by the 
Gini coefficients. In addition, we identified several new variables that could 
potentially influence either measure of programmatic ratings, and may serve as an 
indicator of rater bias. These three variables are (i) the public versus private status of 
the institution, (ii) the number of other highly-rated programs within the institution, 
and (iii) the program's rating from the 1982 NRC study. The first variable tests 
whether raters give higher or lower ratings to programs located in public universities. 
The second new variable, which is calculated as the number of programs at the same 
institution (not counting the program in question) with a scholarly quality rating of 
3.5 or higher, can be viewed as a measure of the institution's overall reputation. With 
this variable, we test the conjecture that peers would rate programs higher if they are 
located in an institution with more highly-rated programs. Finally, the inclusion of 
the lagged program rating was designed to identify the degree to which a program's 
current rating is influenced by its previous rating, holding other factors constant. The 
means and standard deviations for selected variables are shown in Table 1: 
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Table 1 : Means and Standard Deviations (all fields) 



Variable 


Mean 


Standard Deviation 


Scholarly Quality Rating 


3.22 


0.8 


Program Effectiveness Rating 


3.18 


0.7 


Number of Faculty 


29.87 


21.8 


Publications Per Faculty 


6.67 


4.8 


Citations Per Faculty 


40.47 


59.7 


Number Highly-Rated Programs 


10.17 


10.2 


% Public Institutions 


70% 


— 


Correlation of Quality Ratings = 
Number of Programs = 1,809 


+0.96 





Note from Table 1 that, as reported in Goldberger et al. (1995), there is an extremely 
high partial correlation between the two program rating measures that we consider in 
this study. The independent variables that were included in the particular multiple 
regression models reported below included the following measures of program, 
institutional, and student characteristics: 

Program Characteristics: Number of faculty and number of faculty squared 

Publications per faculty member, 1988 to 1992 
Gini coefficient of publications, 1988 to 1992 
% of faculty who are Full Professors 
% of faculty with research support, 1986 to 1992 
Twenty-Nine dummy (0-1) variables for field 

Institutional Characteristics: Public or private institution 

Number of highly-rated programs 
Total library volumes (in 10,000s) 

Total research and development expenditures 
(in $ 10,000s) 

Student Characteristics: % students holding graduate or research assistantships 

% students holding teaching assistantships 
Ratio of graduate students to faculty 
Median years to completion of doctorate degree 

Most of the variables listed above are self-explanatory, with the exception of the Gini 
coefficient. In this application, the Gini coefficient is a measure of the dispersion of 
publications across faculty within a program. As the Gini coefficient becomes larger, 
the publications within a program are said to be more concentrated among a few 
faculty, and vice-versa. Ehrenberg and Hurst (1996b) show that there is a negative 
relationship between the Gini coefficient and the scholarly quality rating of faculty, 
implying that programs with a more even distribution of publications across faculty 
will have higher scholarly quality ratings, holding all else constant. 
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In Table 2, we summarize the results from two of the regression models that 
we estimated explaining how these and other factors influence either the scholarly 
quality rating of programs and the program effectiveness ratings using the pooled 
sample. It should also be noted that we corrected the standard errors in all of the 
regression models for the presence of heteroscedasticity. 

Table 2: Effects of Selected Variables on the Scholarly Quality Rating and Program 

Effectiveness Rating From the 1993 NRC Survey 



Variable 


Effect on Scholarly 
Oualitv Ratine 


Effect on Program 
Effectiveness Ratine 


Historically-Important Factors 


Publications Per Faculty 


+0.03 


+0.02 


Number of Faculty 


+0.02 


+0.02 


(Number of Faculty)^/ 1000 


-0.09 


-0.08 


New Variables 


Number of Highly-Rated Programs 


+0.03 


+0.02 


Public Institution (l=yes, 0=no) 


-0.22 


-0.18 


R-Squared 


+0.75 


+0.71 



All of the estimated coefficients shown in Table 2 were statistically significant at the 
1% significance level. Collectively, it can be seen that the independent variables used 
in our regression model account for approximately three-fourths of the total variations 
in program ratings across institutions. 

The results for the scholarly quality ratings measure were consistent with the 
existing literature, and what one might expect to find a priori. We found that 
programs with higher average publication rates tended to have higher predicted 
scholarly quality ratings, and as shown by Ehrenberg and Hurst, that the size of a 
program has a strong (quadratic) effect on scholarly quality ratings, holding the other 
variables in the model constant. 

Interestingly, we found that these and other factors not only had the same 
directional effect on the program effectiveness ratings, but also had very similar point 
estimates as well. For example, if the faculty in a particular program were to publish 
one more article on average for the entire five-year period, this would raise both their 
predicted scholarly quality rating and program effectiveness rating by 0.03, again 
holding all else constant. This similarity is shown graphically in the chart below: 
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Effects of Selected Variables on Alternative NRC Quality Measures 




Average Publications # of Top Programs Public Inst. 



□ Scholarly Quality 

□ Program Effectiveness 



Independent Variable 



We found much the same story when we looked at the results for the new 
variables that we identified in our analysis. With regard to the institution-specific 
measures that we added to the regression model, the data suggest that raters would 
assign lower ratings to programs when they were housed in public versus private 
institutions. Furthermore, if a program was in an institution with many highly-rated 
programs, raters tended to assign higher ratings to that program, independent of other 
factors that should influence ratings such as publication and citation rates of faculty. 
This is consistent with Fairweather's (1988) suggestion that overall institutional 
quality or prestige plays a significant role in influencing a program's perceived 
quality. Once again, these institutional characteristics had very similar effects on the 
program effectiveness ratings as well. 

Finally, to determine the extent to which current program ratings are driven by 
previous ratings, we reestimated the multiple regression model showoi above after 
including an additional regressor for the mean program rating from the 1 982 NRC 
study. Due to the presence of missing values for these variables, the sample sizes in 
the regression models were smaller than those showoi above (n= 1,020 and 1,016 for 
scholarly quality and program effectiveness, respectively). We found that there was 
evidence of a strong positive relationship between a program’s current scholarly 
quality rating and its rating in the previous NRC study from 1982. This suggests that 
there is considerable inertia in current reputation measures, in that the reputations of 
programs take time to evolve and change. The same result applies to the program 
effectiveness rating. Nonetheless, it is interesting to observe that all of the key factors 
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were still highly significant predictors of the current scholarly quality and program 
effectiveness ratings, even when holding constant each program’s previous rating. 

SUMMARY 

In this study, we have summarized the results from our analysis of the 
programmatic ratings — both for the scholarly quality of faculty and program 
effectiveness in educating scientists — derived by the 1993 NRC survey using 
multiple regression analysis. We found that both reputational ratings are influenced 
by the same set of faculty, student, and institutional characteristics, and are highly 
correlated with each other. Thus, as a general barometer of a program's reputation or 
quality, each of these measures should yield similar insight and rankings. 

The similarity between the two ratings raises the question of whether students, 
faculty, and administrators can gain more insight into the quality of a particular 
program by considering the program effectiveness rating in addition to the scholarly 
quality rating. Our results suggest that in the aggregate there is little “value added” 
from the program effectiveness measure. There are two possible explanations for our 
findings. First, scholarly quality and program effectiveness, while being two separate 
concepts, are influenced in similar ways by the same set of factors. Under this 
scenario, raters have good information on both concepts for a wide variety of 
programs in their field, but the two measures tend to go hand-in-hand; whatever has 
positive benefits for the perceived scholarly quality of faculty likewise has positive 
benefits for the program’s ability to train young researchers, and vice-versa. 

The second possibility is that raters, having poor information on the ability of 
various programs to train young researchers, use their impressions of the scholarly 
quality of faculty to make judgements on the former construct. In this case, the two 
measures — scholarly quality and program effectiveness — essentially represent the 
same concept. If this is true, then perhaps alternative means should be developed for 
securing information on the teaching effectiveness of academic programs. 

Our results also offer some constructive ideas on how a program might strive 
to improve their mean rating, and in the process, their relative rank within their field. 
We have demonstrated that although the program ratings from the NRC study are 
computed from subjective rater opinions, those opinions appear on average to be 
effective at representing the visibility of research output of a program's faculty. 
Therefore, changes within a program that would lead to increased visibility should 
eventually have an impact on the program's rating. For example, initiatives that 
would lead to an increase of, say, one publication per faculty member per year, would 
be predicted to lead to an increase in the mean ratings of approximately 0.15. Such a 
change might be sufficient to move a program up several places in the ratings over 
time. 
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ABSTRACT 

To what extent do regulatory climate differences among states influence the 
satisfaction levels of campus managers? This study first measures the financial, 
personnel and academic dimensions of state regulation, and examines the extent to which 
university and state characteristics have an effect on the regulatory climate and 
administrative flexibility granted to campuses. Second, the research analyzes the 
dimensions of managerial satisfaction and tests the hypothesis that the state’s regulatory 
climate exercises an influence on the satisfaction levels of managers who are in functional 
;areas impacted by state control. An array of organizational, individual, and work climate 
variables are used as controls. An atmosphere of administrative teamwork and 
interpersonal stress appear to exert the strongest positive and negative influences on 

administrative satisfaction. 



BEST COPY AVAILABLE 



161 



160 



ADMINISTRATIVE SATISFACTION AND THE 
REGULATORY CLIMATE AT PUBLIC UNIVERSITIES 



The Research and Policy Problem 

While demands for accountability and control have produced, over the past 30 
years, an increase in governmental regulation, the more recent atmosphere in Washington, 
D.C. and several state capitals reflects a growing consensus that regulation can be both 
costly and unproductive. “Taken by itself, any single action may not be unbearably 
intrusive, but the combined impact of many actions can nearly suffocate an institution" 
(Carnegie Foundation, 1982, p. 65). 

Most organizational behaviorists believe that an increase in monitoring activity 
increases operating costs, both for those doing the monitoring and for those being 
monitored (Downs 1967). Many authors in higher education condemn regulations that 
hamper an institution from adjusting to changing circumstances and needs. A number of 
publications by the Association of Governing Boards (Gardner, et al., 1985), the Carnegie 
Foundation for the Advancement of Teaching (1982), the Carnegie Council on Policies 
Studies in Higher Education (1976), the Carnegie Commission on Higher education 
(1973), and the Sloan Commission (1980) have agreed that over-regulation is wasteful. 

The Carnegie Commission (1973) concluded that campus autonomy has declined 
substantially since the end of World War II. For the nation as a whole, state legislative 
enactments per year increased from roughly 15,000 in the 1950s to 50,000 annually in the 
1980s (Fisher 1988). While not all of these bills affect higher education, Fisher examined 
four representative states and found that nearly half of the higher education laws in this 
century had been enacted in the most recent two decades. However, the nature and 
intensity of these measures differ from state to state due to the varying nature of state 
history, structure, culture, law, educational standards, and political tradition. In heavily 
regulated environments, public universities are treated like "state agencies" and have less 
flexibility in personnel, financial and academic matters. Campuses in other states are 
relatively autonomous, and considered to be "state aided" (Curry & Fisher 1986). 

The 1982 study by the Carnegie Foundation reported that state officials fail to 
reward efficient leaders, and that campus managers feel caught in a bureaucratic web that 
demands accountability, but provides few incentives for responsible management. 
Administrative dissatisfaction with over-regulation is worth examining because the job 
satisfaction literature indicates a strong connection between satisfaction and productivity 
(Vroom 1964, Porter and Lawler 1968). 

Much has been written about government regulation of public higher education, 
but there are few empirical studies that analyzed the relationship of state regulation to 
various campus and state characteristics except Volkwein's National studies in the 1980s, 
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and more recently in the 1990s (Volkwein 1986, 1987, 1989; Volkwein & Malik 1996, 
1997). While these studies found little relationship between measures of 
regulation/autonomy and other university characteristics and behaviors, they offered the 
hypothesis that state controls have a negative impact on administrative satisfaction. No 
studies in higher education have attempted to measure the impact of state regulation on 
administrators job satisfaction, despite the linkages in the research literature among 
satisfaction, performance, stress, and turnover (Austin & Gamson 1983, Glick 1992, 
Hagedom 1996, Smart 1990). Researchers have shown a consistent inverse relationship 
between worker job satisfaction and turnover rates (Cotton & Tuttle 1986, Glick 1992, 
Mobley et al. 1979, Smart 1990). 

Thus, there are multiple reasons to be concerned about state regulation and 
campus autonomy. First, the empirical literature on state regulation and its impact on 
campuses is sparse. Second, unless its benefits can be demonstrated, regulatory activity 
by the state appears to be unproductive and wasteful of taxpayer resources. Third, 
regulatory activity may reduce managerial job satisfaction which in turn increases 
turnover and lowers organizational productivity and adaptation. Since educational 
institutions, along with the rest of society, need to become more productive, we need to 
remove as many obstacles to increased productivity as possible. Since the States as well 
as the Federal government provide much financial support to this large higher education 
industry, they have a stake in knowing the impact of regulatory actions. 



Purpose of the Study and Conceptual Framework 

To what extent do regulatory climate differences among states influence the 
satisfaction levels of campus managers? This study addresses this question first by 
measuring the administrative and academic dimensions of state regulation, and examining 
the extent to which university and state characteristics have an impact on the regulatory 
climate. Second, the research examines the perceived work environments and individual 
characteristics of administrative managers whose positions are impacted by state control. 
With responses from almost 1000 managers at 100 universities, we analyze the 
dimensions of managerial satisfaction and test the hypothesis that the state’s regulatory 
climate exercises an influence on satisfaction levels. We conducted this study by merging 
a variety of theoretical perspectives from the research literature: organizational theory, 
structural/fiinctional perspectives, the literature on university autonomy, and theories of 
employee satisfaction. 

Contemporary organization theory stresses the role of the organization’s 
environment as a crucial influence on the life of an organization, its structure, and its 
activities. Contingency theory (Lawrence and Lorsch, 1967), the natural selection model 
(Aldrich, 1979), and the resource dependence model (Pfeffer and Salancik, 1978) all 
focus on the external environment. Volkwein (1986, 1989) found support for the 
resource dependence model in his studies of university quality. Public universities are 
viewed as complex, loosely-coupled organizations, and their relations with state 
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governments form a critical component of the external climate within which they pursue 
their goals. Scholars generally divide an organization’s environment into economic, 
political, social, and technological dimensions. 

Structural/functional perspectives from the literature on organizations and 
bureaucracies encourage researchers to attend to those variables that reflect the influence 
of organizational structures (Hall, 1991). Studies of colleges and universities, as 
particular types of organizations, have shown that campus mission, size, wealth, and 
selectivity exert significant influences (ranging from small to large) on a variety of 
college outcomes (Pascarella and Terenzini, 1991). Volkwein (1986, 1989, 1995) has 
demonstrated that a variety of university characteristics tend to cluster together along the 
dimensions of mission, size, financial support, and complexity. In their summary of the 
research literature, Austin and Gamson (1983) call for the incorporation of such factors as 
institution size, control, and selectivity into future studies of administrative satisfaction. 
Hagedom (1994) notes the importance of university and department quality as influences 
on satisfaction. 

The literature on university autonomy and state control suggests that there are 
multiple dimensions. Berdahl (1971) distinguishes between “procedural” and 
“substantive” autonomy. Levy (1980) and the Carnegie Foundation (1982) identify three 
important areas of university autonomy: financial or budgetary, personnel or appointive, 
and academic. In his studies of state regulation and campus flexibility, Volkwein (1986, 
1987, 1997) found that the financial and personnel dimensions collapsed into a single 
administrative factor, leaving academic and administrative as the two autonomy 
dimensions. 

The job satisfaction literature provides several conceptual frameworks that 
guide the development of this study and its measures. Maslow's hierarchy of needs is 
used by most management experts to argue in favor of worker autonomy and 
organizational flexibility (Boons & Kurtz, 1992). Herzberg's Two Factor Theory (1959) 
draws our attention on the one hand to intrinsic job content factors (such as feelings of 
accomplishment, recognition, and autonomy), and on the other hand to extrinsic job 
context factors (such as pay, security, and physical working conditions). 

For instance, several researchers have identified the opportunity for independent 
thought and action, feelings of accomplishment, opportunities of growth and 
development, and self-esteem as indicators of intrinsic rewards (Olsen, 1993; Austin and 
Gamson 1993; Hackman and Lawler, 1971). On the extrinsic dimension, Kalleberg 
(1977) demonstrates that job satisfaction is positively affected by job reward factors such 
as pay. Hagedom’ s (1994) causal model of satisfaction in academe shows satisfaction 
with salary, total work hours, and perceived support of colleagues as directly influencing 
perceived level of stress which, in turn, directly effects satisfaction. At any rate, there is 
general agreement that job satisfaction is multi-dimensional and influenced by a complex 
array of personal and situational circumstances (Austin & Gamson, 1983; Hoppock, 

1977; Mumford, 1972; Bmce and Blackburn, 1992). 
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Consistent with research in other organizations, studies of managers in colleges and 
universities suggest that a variety of personal and organizational variables exert potential 
influences on their job satisfaction. Among these personal characteristics are age (Austin, 
1985; Lee & Wilbur, 1985; Solomon & Tierney, 1977), sex (Austin 1985; Hagedom 
1996), level of education (Martin and Shehan 1989), length of service (Bamimdo & 
Kopelman 1980), administrative rank (Austin 1984), and administrative area (Click, 
1992). 



The organizational characteristics that may influence managerial satisfaction 
include organizational mission, size, wealth, complexity, and quality (Austin & Gamson 
1983, Hall 1995). Several studies also show that a variety of work related variables exert 
positive and significant influences administrative satisfaction — a supportive 
organizational culture, teamwork, relationships with colleagues and superiors, worker 
autonomy, and self-fulfillment (Berwick 1992, Bensimon & Newman 1991, Austin & 
Gamson 1983, Boone 1987, Lawler 1986, Rigg 1992). 

Negative influences on satisfaction are produced by job and workload stress (Blix 
& Lee 1991, Click 1992, Olsen 1993). Because of the connection between the two, job 
stress variables are almost always included in studies of job satisfaction (Blau 1981, 
Hagedom 1996). Moreover, there are multiple sources of stress related to roles and 
responsibilities, tasks and workload, social and interpersonal relations, and personal 
health (Blix & Lee 1991, Smith et al. 1995, Sullivan & Bhagat 1992). Thus, in this study 
we control for the influences of outside stresses like personal and family health problems, 
personal and family financial problems, and interpersonal difficulties with colleagues and 
superiors. Many in higher education argue that the regulatory climate influences the 
satisfaction levels of campus managers and over regulation produces a controlled work 
environment (Atwell 1985, Carnegie Foundation 1982, Curry & Fisher 1986, Mingle 
1983, Newman 1987). Solomon and Tierney (1977) and Smart and Morstain (198 ) 
found higher satisfaction among college administrators reporting a work environment that 
is appealing and challenging and consistent with their preferred responsibilities. Thus, 
we complemented our measures of actual state control with measures of the perceived 
regulatory control and work environment. 

Research Methods 



We created the analytical database for this study drawing information from a wide 
range of different sources. Information on state characteristics was assembled from U.S. 
Census data, the public administration literature, and a survey of state regulation and 
management flexibility practices. The organizational and financial characteristics of 
universities came from NCES/IPEDS, from the National Research Council study of 
doctoral programs(1995), from the Graham and Diamond research at Vanderbilt (1996), 
and from the guidebook information contained in Barron's and US News. The 
information about the backgrounds, perceptions, personal stress, and satisfaction levels of 
individual administrators was obtained on a survey that was administered by cooperating 
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campus officials to a population of managers with 12 specific job titles that we 
hypothesized would be impacted by state control and campus flexibility. 

Target Population, Data Sources, & Variable Summary 

This study solicited responses from 144 public universities in the United States 
classified by the Carnegie Foundation as Research I & II or Doctoral I & II. 122 
universities in 50 states agreed to participate in the study, and their Presidents and 
Chancellors designated a campus contact person to assist the researchers with data 
collection. Each campus received two types of surveys. The first survey (of management 
flexibility and state regulation) contains 47 questions about financial, personnel, and 
academic transactions. This first survey was returned by 122 public universities. 

The second survey of administrative satisfaction and stress was administered to 12 
managers (ranging from vice presidents to directors) on each campus who hold positions 
that in theory are effected by the regulatory climate. This second survey contains 7 
questions about the respondent's background, and 44 items assessing their satisfaction, 
stress, and working conditions. Using procedures to ensure respondent anonymity, we 
eventually received 995 satisfaction surveys from 100 cooperating universities. The 
number of administrative respondents by rank and functional area is shown on the 
following page. While not every campus had all 12 managerial titles, we received an 
average of 10 satisfaction surveys per campus. 

The attached diagram of variables in this study shows the measures that we 
assembled from these various sources. A more complete description of the methodology 
and data reduction steps for the state attributes, campus characteristics, and measures of 
regulation and flexibility is contained in Volkwein and Malik (1997, forthcoming). 

State attributes: We incorporated into the database thirty-seven state characteristics that 
fall into three broad types of measures: economic (e.g., per-capita income, tax capacity, 
support for higher education, poverty rate), social/demographic (e.g., population 
attributes, mobility, ethnic and age profiles, education levels), and political/bureaucratic 
(e.g., relative size of state government, governor's power, political culture, voting 
patterns). State characteristics data were obtained from NCES "State Higher Education 
Profiles," from U.S. Census Data, from the Center for the Study of the States, and from 
other higher education and public administration literature. Data reduction in the form of 
principle components analysis and scale building produced the five variables shown in 
the diagram — size, wealth, political culture, mobility, and public sector strength. 

Campus characteristics: Based on the organizational literature (Hall 1991, Volkwein 
1986), and our own confirmatory factor analysis, the 42 campus characteristics separate 
into four broad categories: organizational size, financial support, 
mission/complexity/diversity, and quality/selectivity. The measures reflecting 
organizational size, wealth, and quality formed tight scales, but the complexity measures 
are not highly enough correlated to form a common factor. The measures of size, wealth. 
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and complexity we extracted from IPEDS and other data supplied by the National Center 
for Education Statistics. Variables reflecting faculty and student quality have been 
constructed from the national survey of doctoral program quality by the National 
Research Council, from the data assembled by Graham and Diamond (1996), and from 
the student and campus data reported in Barron ’s and U.S. News. In addition, we 
obtained AAUP salary data and within-campus Directory information about 
administrative officers and colleges and schools. From the campus administrative 
flexibility survey we obtained information about levels of employee unionization, 
flagship status, and constitutional autonomy. 

State regulation/autonomy: Rules, legislation, and procedures, prescribed by the states 
to control academic, financial and personnel transactions of universities, are measured by 
campus responses to survey items and scales originally developed by Volkwein (1986, 
1987, 1989), but updated and enhanced for this study (see Volkwein & Malik 1997). The 
survey collected responses to questions about 47 types of flexibility and control. Among 
the 26 survey items about financial and personnel matters, we found five dimensions of 
financial and personnel flexibility that are so highly correlated we combined them into a 
single scale of administrative flexibility. These items reflect various aspects of campus 
flexibility over managing budgets and revenues, expending funds, setting tuition, and 
appointing personnel without external approval. Regarding the 21 survey questions about 
academic flexibility, we found six dimensions that encompass all the academic survey 
items and form a common scale. These items reflect an institution’s autonomy over 
academic programs, degree requirements, standards, and departments, as well as the 
institution’s freedom from state imposed accountability requirements. 

Administrator Characteristics: Based on the literature, we asked respondents to 
indicate their age and length of service, sex, highest degree, academic rank (if any), 
administrative rank (we imposed a hierarchy), and functional area (academic affairs, 
business and finance, human resources management, IR and planning, student services, 
and other) 

Managerial Satisfaction, stress, and work climate: Using an anonymous survey 
distributed by the campus contacts to the 12 managers with job titles that we 
identified, we assess respondent satisfaction and stress in a variety of dimensions 
suggested in the occupational satisfaction literature. The 25 satisfaction items on the 
survey use a 5-point Likert type scale from very satisfied to very dissatisfied. Our 
principle components and data reduction techniques produced scales of intrinsic 
satisfaction, extrinsic satisfaction, satisfaction with work conditions, and 
relationships with others. Their alpha reliabilities range from .76 to .90. We use a 
single item to express overall satisfaction. 19 other questions use a 5-point scale to 
elicit responses to questions about work climate and sources of stress (l=none, 
3=moderate, 5=extreme). These yielded scales reflecting perceptions about 
regulation, control, teamwork, workload, adequacy of funding and facilities, and 
stress from health, financial, and interpersonal situations. 
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Analytical Procedures 



We concluded data collection during 1995, and in early 1996 completed a series 
of data reduction steps and began the analysis. The data assembled from these various 
sources were merged into a database and manipulated using SPSS. We conducted 
principle components analyses to confirm the dimensions of state attributes, university 
organizational characteristics, regulation/flexibility, perceived work climate, and 
administrative satisfaction. We used a series of OLS Regression equations to test the 
hypothesis that regulatory climate differences among states influence the satisfaction 
levels of campus managers. 



RESULTS 



The attached Flexibility Grid shows our classification of the 50 states on the 
relative amounts of administrative and academic flexibility that they give their 
universities. For each of the two flexibility dimensions, we took the separate factor 
scores from the principle components analysis and added them to produce a single overall 
scale for academic flexibility and one for administrative. Each state scoring one standard 
deviation or more above the mean is classified “high flexibility;” each scoring one 
standard deviation or more below the mean is “low.” The rest are “medium.” The 
universities with the greatest flexibility and the least state oversight on both dimensions 
are in the high/high box. States like Ohio, New Mexico, and Michigan enjoy above 
average autonomy on both dimensions. On the other hand, Illinois and Maryland are the 
only states in the low/low box, indicating that universities in Illinois and Maryland have 
comparatively less autonomy and more state oversight than their peers in this study. 
Reflecting the independence of these two flexibility dimensions, some states rate 
high/low while others rate low/high. For example. New York and Virginia are relatively 
high on academic and relatively low on administrative flexibility. On the other hand, 
Colorado and South Carolina experience the opposite condition: high administrative, low 
academic. 

The question is, do these differences among states translate into variations in the 
levels of administrator satisfaction? To address this question, we first examined the 
correlations among the relevant variables in the database, and second, calculated a series 
of OLS Regression models. The attached Correlation Matrix shows the zero-order 
correlations among the two objective measures of campus autonomy (1 administrative, 2 
academic), the perceived measure of external regulation (3), the three work environment 
stresses (4-6), and the five measures of satisfaction that we developed from the survey (7- 
11). The correlation matrix shows that each of the two objective measures of 
autonomy/flexibility (variables 1-2) are significantly correlated with respondent 
perceptions about external regulation, but the coefficients are only -.09 and -.35. The 
measure of administrative autonomy is additionally correlated with perceptions of a 
controlled work environment (-.09). However, neither of the first two flexibility 
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measures is significantly correlated with the measures of workload and interpersonal 
stress (5-6), nor the measures of satisfaction (7-1 1). 

Interestingly, the measure of perceived external regulation (3) is significantly 
correlated with perceptions about a controlled work environment(4) workload stress(5), 
and interpersonal stress(6), but not with the satisfaction measures(7-l 1). However, the 
workload, interpersonal, and controlled work environment measures are negatively 
correlated with all five satisfaction measures. This suggests the possibility of indirect 
influences at work on satisfaction through the work environment. 

Last, the correlation matrix shows that each of the five satisfaction measures (7- 
11) are strongly correlated with each other — coefficients range from .34 to .69. Most 
strongly a correlated with overall satisfaction is the scale of work conditions. 

We next developed a series of stepwise, pairwise, OLS regression models by 
entering the state characteristics, then the campus characteristics, then the regulation 
measures, then the administrator attributes, and finally the work climate variables. Until 
we entered the work climate variables the models explained trivial amounts of the 
variance in the five dependent satisfaction measures. The results of the final multivariate 
analyses are shown in the Regression Table which shows only the significant beta 
weights and R-square values. 

When all the state, campus, and administrator variables are included in the 
stepwise regression procedure, the first column shows that four measures explain 28 
percent of the variance in overall administrative satisfaction: This is a single item, rather 
than a scale, and the most important variables are the positive influence of administrative 
teamwork (.26), and the negative impacts of interpersonal work relations (-.31), workload 
stress (-.11), and perceptions of an overly controlled work environment (-.10). 

Three of these four variables exert similar influences in the next column of the 
table — the regression model for the scale of intrinsic satisfaction. (This eight-item scale 
reflects feelings of accomplishment, autonomy, creativity, initiative and challenge.) With 
an R-square of .31, the most significant influences are exerted by interpersonal stress (- 
.31), teamwork (.27), and a controlled work environment (-.18). Significantly less 
intrinsic satisfaction was reported by managers in human resources (-.09) and 
significantly more by those in states with a strong public sector commitment (.07). 

The analysis of extrinsic satisfaction (a three item scale of salary & benefits, 
opportunities for advancement, and future income potential) explains only 14 percent of 
the variance. Influential variables include undergraduate quality (.12), administrative 
rank (-.18), teamwork (.14), inadequate funding (-.11), and interpersonal stress (-.15). 
Perhaps the only surprise here is that the higher the administrative rank, the less 
extrinsically satisfied. 
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Three variables explain 41 percent of the variance in the scale of work conditions. 
(These five items reflect satisfaction with hours, work pressure, job security, and 
organizational politics). Large negative influences are exerted by workload stress (-.42) 
and interpersonal stress (-.29). Again, teamwork has a positive influence (.19). 

The regression model for satisfaction with work relationships (5 item scale 
reflecting relationships with colleagues, faculty, and students) only explains 20 percent of 
the variance. Once again, interpersonal conflict exerts a negative influence (-.28) and 
teamwork exerts a positive influence (. 1 8). Undergraduate quality (.07) and having 
academic rank (.16) constitute other positive influences, while the perceived regulatory 
climate is a negative influence (-.08). 

Looking across all five models, it is clear that the largest, most consistent, and 
opposite influences on administrative satisfaction are interpersonal stress and teamwork. 
Not only do they explain most of the variance in expected aspects of satisfaction (intrinsic 
and work relationships), but also in the less expected realms of extrinsic rewards and 
work conditions. Their influence is so pervasive their influence on overall administrative 
satisfaction is two or three times greater than perceptions about workload stress and about 
an overly controlled work environment. 



Discussion and Conclusions 



This topic is significant for a number of reasons. There has been much public 
discussion about the virtues of deregulation and decentralized, customer-based 
management. This study documents the 1995 status of each state’s regulatory practices 
toward public universities and examines the evidence for a link to the satisfaction of 
CEhnpus managers. Although finding little relationship between the regulatory climate for 
public universities and a variety of other campus measures, Volkwein (1986,1987,1989) 
raised the possibility of a connection to managerial satisfaction. The empirical literature 
on state regulation and its impact on campuses is sparse. No studies in higher education 
have attempted to measure the impact of state regulation on administrators job 
satisfaction, despite the linkage in the organizational literature among managerial 
satisfaction and performance and turnover. 

Drawing from contemporary organizational theory and building upon the literature 
on satisfaction, this study examines the nature of state control and university autonomy. 
Using a combination of campus survey data, follow up telephone interviews, and 
information from national sources, we constructed a database containing an array of 
measures reflecting campus characteristics, state attributes, the regulatory climate for 
public universities, and the survey results from a selected set of campus managers who 
responded anonymously to questions about their personal characteristics, work 
environments, and levels of satisfaction. 
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First, we confirmed the existence of two strong and relatively independent 
dimensions of campus autonomy: academic and administrative. The academic dimension 
is composed of factors reflecting campus flexibility regarding programmatic actions, 
setting standards and policy, degree requirements, restructuring departments, and 
responding to accountability requirements. The administrative dimension reflects campus 
flexibility regarding managing revenues, setting tuition and fees, classifying and 
appointing personnel, budgeting and expending funds. Such a two-dimensional 
framework is consistent with Berdahl’s procedural and substantive autonomy and with 
Volkwein’s studies in the 1980s that identified academic and administrative autonomy 
factors similar to these. 

Second, we assembled an array of data that reflects the economic, demographic, 
social, and political characteristics of each state and found little evidence of a connection 
between these characteristics and either the regulatory climate for state universities or the 
satisfaction levels of their managers. There was a slight tendency for large states to be 
more controlling but this accounts for only 12 percent of the variance in administrative 
flexibility and none in academic flexibility. (Volkwein & Malik 1997) 

Third, we also found little evidence of a connection between state control and 
campus characteristics. The amount of regulation is unrelated to the age, size, stature, 
mission, complexity, and cost of public universities. Out of 12 variables in two 
regressions, only one was significant (the percent of minorities) and it explains a mere 
seven percent of the variance in administrative flexibility and none in academic 
flexibility. (Volkwein & Malik, 1997) 

Fourth, we examined the evidence for a relationship between the university’s 
regulatory climate on the one hand and several dimensions of satisfaction on the other. 

We find that most state and campus characteristics, including the regulatory climate, have 
little direct relationship to administrator satisfaction. One exception is that intrinsic 
satisfaction is higher in states that give a priority to public sector programs and 
universities. Another exception is that undergraduate quality (freshman selectivity) 
measures are significantly associated with extrinsic satisfaction and satisfaction with 
work relationships. We do find that the actual regulatory climate (as measured by our 
data collected from institutions and states) has statistically significant but low correlations 
with perceptions about overly regulated and controlled work environments. These 
perceptions in turn are correlated with perceived work stress and satisfaction. However, 
most of the correlations are relatively low, ranging from .09 to .24. 

While many in higher education believe that there is a connection between 
campus autonomy and administrative satisfaction, we find only the thinnest of 
evidence to support such a claim. In order to explain the low relationship between 
campus autonomy and job satisfaction, we need to consider the complexity of the 
satisfaction phenomenon. Job satisfaction is an overall measure of one’s feelings 
toward their work (Kalleberg, 1977). Thus, it is possible for a worker to be 
dissatisfied with one aspect of their job, such as the salary, but on the other hand, be 
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very satisfied with alternate aspects, such as interpersonal relationships or elements 
they find intrinsically satisfying. Workers may weigh factors against one another and 
reach a combined satisfaction score in which positive feelings balance out negative 
feelings. Thus, although we did not necessarily expect to find evidence of a direct 
relationship between the regulatory climate and workers’ overall job satisfaction, we 
did expect to find evidence of campus autonomy affecting some of the sub- 
components of job satisfaction. 

Our analysis suggests that external regulation has no direct influence on any of 
the five satisfaction measures. Rather, the regulatory climate may have indirect but 
small influences on the work environment and on interpersonal stress. However, 
workplace relationships and an atmosphere of teamwork have highly positive impacts 
on most measures of satisfaction. This finding is consistent with Hagedom’s (1996) 
recent research showing that interpersonal relationships positively influence job 
satisfaction and also lessen job related stress. 

The important and statistically robust finding in our study is the solid and 
consistent connection between every measure of administrative satisfaction and the 
human relations aspects of university administration. Interpersonal work stress exerts a 
consistent negative influence on satisfaction and an atmosphere of teamwork exerts a 
consistently positive influence. These results strongly support those theories and 
administrative practices which accent the importance of interpersonal relationships in 
creating positive work environments and organizational climates that are satisfying to 
managers. 
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List of Satisfaction Survey Recipients 



1) Chief Academic Officer (Provost, Vice President, Vice Chancellor) 

2) Chief Business & Finance Officer 

3) Dean of largest college/school (non-medical) 

4) Managers regardless of title that have responsibility for the following 
administrative offices: 

a) Registrar 

b) Institutional Research/planning 

c) Personnel & Human Resources 

d) Budget 

e) Purchasing 

f) Controller/ Accounting 

g) Payroll 

h) Physical Plant/Facilities 

i) Financial Aid 



Administrative Respondents by Rank and Functional Area 



Administrative Rank or Equivalent 



Division 


Vice Pres./ 
Provost 


Dean/ 
Assoc. Vice 
Pres. 


Assistant 
Vice Pres. 


Director 


^^Assistant 

to” 


Total 


Academic Affairs 


83 


91 


6 


2 


2 


184 


Business/Finance/Admin 


59 


21 


25 


44 


11 


160 


Student Services 


3 


8 


7 


138 


7 


163 


Planning & IR 


4 


1 


3 


53 


5 


66 


Personnel/Human Res Mgmt 


12 


34 


29 


229 


34 


338 


Other 


2 


10 


16 


53 


3 


84 


Total 


163 


165 


86 


519 


62 


995 
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Flexibility Grid 



Administrative Flexibility 



High 



Academic 

Flexibility 



Medium 



Low 



Low 


Medium 


High 






WY, MI, 






NM, AL 


FL, NY 




DE, OH, 


RI,MA 


NJ, OR 


VT, PA, 


VA 


AKWV 


ND,IA 


KS,TX 


ME, LA 


KY, AR, 


NC, CA 


WA, MT 


IN, CT 


WI 


UT, NB 


NH 




OK, NV 






AZ, MO 




MD, IL 


MN, TN, 


CO, SC 




MS, GA, 






SD, HI 






ID 





er|c 



180 



179 




21 



O 

O 



2| 



21 



CM 



o o 
o 



o 

o 



r-- 

m m 



O m On ^ 
O Tf Tf Tf 



vH 

CX) 



001 



z 

08 

C8 

> 



t-| 



.5 

yi 

CQ 



^oi 






Q> 

C8 

E 

C 

o 

i2 

3 

D£ 

Q> 

cc 



f^l 



* * * * 
')!■««« 

O On (N un ^ 
O VO Tf Tf 



O 

o 



# # # # # 

-M'-M'-M-** 

* * * * * 

00 NO <N On 

rr Tj- (N m >— 



« « « « « « 

* # « # # # 

« ■» « ■» « « 

o ^ m NO NO 

o fN ^ ^ 

■ I* i’ i’ I* I I 



O 

o 



(N 



* 

* 

* 



* * * * * 

*•»#** 

(N m m ^ m 

^ ^ ^ in ^ 

i’ i' i' i’ i' 



* * 
« * 
* * 
o (N r- 
o — — 



* 

* 




On ^ O m 



I I I I 



-M- 

r" 

o 



o 

o 



On (N 

(N O O p 



•» 

1/^ m m NO oo 
o o o o o 



I I I I I 



nl 






o 

o 



O oo 
o •— 



On NO NO to fN 
O O O O p 



NO m 1— I 1— I 

o O o o o 



un On O so NO 

m O O o O 



« ^ m (N (N 

o O p o 



« X 

fli 



u 

> 

s 

-2 

3^ 

01 

or 

cc 

Ci 

■w 

3 

I/) 



C <D 

S > 



0 ^ 

2 J 2 

3 G 

< Is S 

c/) C 

a G 3 

1 ^ - 

cj -;2 



o 


G 


3 


O 


E 




U 


3 

W) 


u 


0> 

Od 


o 





^ P 2 -S 



o 

s 

X o 
W U 

fO lO 



o 



is 

C/3 

*3 

S.ii 

CO 

u 3 

0> 

s. ^ 

cr « 



G 

O 



c 

o 



^ u 

*3 



^ ^ ^ „ G .G 

4> cx ‘-G s> ‘P is 

G X U > S X 

Z w ^ O 2 ua 

NO r- .SS 00 On O 



04 

Z 

c/5 

G 

s 



3 

C/3 



o 

CO 



“3 



BEST COPY AVAILABLE 



Regression Table 

Regression Beta Weights for Administrative Satisfaction Measures With 
_ Characteristi cs of States, Ca mpuses, and Administrators 



independent Variables 


Dependent Variables (N=996) 


Overall 

Satisfaction 

(1 Q) 


Intrinsic 

Satisfaction 

(8 Qs, @=.90) 


Extrinsic 

Satisfaction 

(3 Qs, c®=.77) 


Work 

Conditions 

(5 Os. ®=.76) 


Relationships 
with Others 

(5 Qs, @=.82) 


State Characteristics 


State size 












State wealth 












Public sector strength 




.07* 








State mobility and growth 












State political culture 












Campus Characteristics 




Campus size 












Campus wealth 












Faculty quality 












Undergrad Quality 






.12*** 




.07* 


Has Medical/Hospital 












Has Agricultural college 












Flagship 












Constitutional recognition 












Campus age 












Pet. Students in dorms 












Campus rural environment 












Employee unionization 












Pet Minority students 












Autonomy/Flexibility 




Administrative 












Academic 












Administrator Characteristics 




Age 












Female 












Highest degree 












Academic Rank 










.16*** 


Administrative rank 






-.18*** 






Division 


Academic Affairs 












Business & Finance 












Human Resource Management 




-09** 








IR & Planning 












Student Services 












Perceived Work Climate 


Perceived Regulatory climate 










-.08^ 


Controlled Work Environment 


-lO** 


-.18*** 








Perceived Adm. teamwrk/commitmt 


.26*** 


21*** 


.14*** 


19^** 


.18**^ 


Sources of Stress on individ. adm. 


Pressure of workload/time 


-A\** 






-.42*** 




Personal Health/financial 












Campus funding/facilities 












. Jntemersonal Relations 


•A\*** 


-2\*** 


-.15*** 




•2S*** 


frotfli = 


.IS*** 


.2\*** 




1 ,20*** 



Significance levels ♦ =* ^.05 Non Significant Bela Weights not Shown 



♦♦-<01 

♦♦♦-<.001 
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A STUDY OF INSTITUTIONAL AUTONOMY IN THE 
COMMUNITY COLLEGE AND ITS RELATIONSHIP TO 
ADMINISTRATIVE JUDGEMENTS OF INSTITUTIONAL 

EFFECTIVENESS 

Anita C. Voogt 

Associate Dean of Academic Affairs 
Dutchess Community College 



Purpose 

The dramatic growth and increased costs of higher education since World 
War II have brought growing concern about the fiscal accountability of public 
colleges and universities. Government provides substantial financial support to 
public higher education; along with its increased dollars has come increased 
government regulation which, many contend, has significantly eroded the authority of 
higher education institutions (Carnegie Foundation, 1982; Millett, 1984; Burke, 1994). 
While recent government regulatory trends are making higher education increasing 
accountable for student academic achievement (Barr, 1995), educators claim that 
excessive government controls on fiscal and personnel matters adversely affect the 
ability of higher education to fulfill its academic mission effectively (Mingle, 1983; 
Millett, 1984). Limited empirical evidence exists, however, to support these claims. 

After years of reported local government interference in the operation of the 
community colleges of the State University of New York (SUNY), in 1988, New 
York State enacted the "Plan C" legislation intended to increase the autonomy of the 
SUNY community colleges in the management of their fiscal affairs, and to create a 
uniform statewide fiscal operating system. This study examined the nature and extent 
to which the autonomy of the SUNY community colleges changed five years after the 
legislation, and explored various factors and political action strategies thought to 
influence institutional autonomy. Additionally, the study explored the perceived 
relationship between autonomy and selected measures of institutional effectiveness. 

Conceptual Framework 

Institutional Autonomy: Institutional autonomy refers to the degree to which 
public higher education can govern itself without external controls (Dressel, 1980). 

In practice, government can exert controls on public higher education through 
procedural restrictions on items such as salary and personnel classification standards, 
and transfers between budget categories (Fonte, 1993). Proponents of government 
regulation claim that such procedural controls do not adversely affect the substantive 
academic core of the institution (Berdahl, 1971), and that government, as a major 
funding source and the authorized taxing agency, has the responsibility to require of 
higher education accountability in its fiscal matters (Haberaecker, 1987). 

Opponents of government regulation, on the other hand, argue that the 
cumulative effect of such controls exceeds financial and personnel matters, and 
infringes on the authority of educators in academic matters (Damowski, 1989). Such 
controls, they claim, reduce campus efficiency, adaptability, and possibly academic 
effectiveness (Dressel, 1980). Furthermore, the university is thought to be a unique 
social institution whose independence is crucial if society is to be well served 
(Carnegie Foundation, 1982). 
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Organizationally, if public colleges and universities are viewed as "open 
systems" influenced by the elements that surround them, their regulatory relationships 
with government form an important feature of the external environment. As such, the 
Resource Dependence Model (Aldrich & Pfeffer, 1976), provides a useful framework 
through which these regulatory relationships and their subsequent effect on the 
autonomy of the institution can be examined. 



Factors and Strategies: Other than the budgetary relationships with the 
external environment, this study examined the correlation between autonomy and the 
following three clusters of variables that, based on the work of Volkwein, (1987), 
Fonte, (1988), and Caswell, (1988), are also thought to be correlated to autonomy. 



Cluster I: Institutional Characteristics - The Age, Size, Mission, Historical 
Development, and Level of Resource Dependence of the college. 

Cluster II: Contextual Variables - Location/Socio-Economic Factors, 
County Government, Perceived Quality and Importance, and State Statutes 

Cluster III: Political Action Strategies - Grass Roots Advocacy, Media 
campaigns. Liaison with State Agencies, Political Action Committees 



Institutional Autonomy and Institutional Effectiveness: Conceptually, 
issues of autonomy is related to institutional effectiveness through the governance 
function of the institution. Governance is the means by which academic leaders make 
critical administrative decisions that influence organizational culture and climate. 
Culture and climate, in turn, potentially affect academic quality and student academic 
achievement (Peterson, 1986). 

Institutional Effectiveness is a complex, multi-dimensional construct that is 
difficult to define and, hence, difficult to measure. Limited empirical evidence exists 
to support claims of the asserted positive relationship between autonomy and 
effectiveness (Volkwein (1986). Using the four effectiveness dimensions of The 
Integrated Model of Institutional Effectiveness (Parsons, 1967), as described below, 
this study explored the correlation between autonomy and a range of effectiveness 
indicators not yet included in the empirical evidence. 

Dimension I: Adaptation - the institution's efforts to accommodate to its 
environment; measured in this study by the nature and extent of curricular innovation. 

Dimension II: Goal Attainment - the setting and achieving of institutional 
goals; measured in this study by Academic Achievement (graduation and attrition 
rates); and Resource Acquisition (funding). 

Dimension III: Integration - the institution's ability to maintain solidarity; 
measured in this study by the perceived administrative supportiveness of the colleges' 
teaching and learning function. 

Dimension IV: Latency - the institution's ability to create and to preserve its 
unique values and identity; measured in this study by the motivational climate and the 
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perceived satisfaction with the institution of the faculty, administrators, and students. 



Methods and Data Sources 

Measures of Autonomy: This study defines autonomy as the degree to which 
theiSUNY community college is free from fiscal and persoiuiel controls exerted on it 
by the local government (referred to as the local sponsor). The sources of autonomy 
data were two unpublished Plan C Surveys of the thirty SUNY community colleges, 
one conducted in 1988 just prior to the legislation, and then repeated in 1994 (Hankin, 
1988, 1994).. The surveys were three part, forced-choice, "yes-no" questioiuiaires on 
fiscal, personnel, and operational autonomy matters. Autonomy levels were measured 
according to regulatory scales developed by Garms (1977), Curry and Fischer (1986), 
Volkwein (1986), and Fonte (1988) in six areas: budgeting flexibility; budget form; 
expenditure oversight; tuition and revenue control; local authority; and persoiuiel 
administration. The survey data was analyzed to determine the nature and extent of 
the change in fiscal, personnel and operational autonomy for three groups: 

Group I: The Thirty SUNY Community Colleges - This analysis shows the 

change in autonomy, system-wide, and identifies those colleges which, in 

1994, had the higher and lower levels of autonomy. 

Group II: The (previously) Plan A and Plan C Colleges - In 1988, twenty-five 
community colleges operated under Plan C; five colleges operated under a more 
restrictive fiscal system, referred to as Plan A, and converted to Plan C as a result of 
the legislation. This analysis compares the change in autonomy between the more 
restricted Plan A colleges with that of the more autonomous Plan C colleges. 

Group III: The Higher and Lower Autonomy Colleges - This analysis com- 
pares the differences in autonomy between the five higher and five lower autonomy 
colleges. These colleges comprised the sample that was studied in regard to the 
factors and strategies thought to be correlated to autonomy, and the perceived 
relationship between autonomy and selected measures of institutional effectiveness. 

Factors, Strategies, and Measures of Institutional Effectiveness: Qual- 
itative data, and where possible, quantitative data was also gathered on each of the 
three cluster variables and four effectiveness dimensions previously mentioned. 
Qualitative data was gathered from one-on-one structured inter-views with the college 
president, chairperson of the board of trustees, and the academic.and administrative 
deans at each of the ten colleges in the sample. Quantitative data was gathered from 
selected SUNY and college reports. 

Comparing the higher and lower autonomy colleges, the qualitative data was 
analyzed to determine the degree to which autonomy is perceived to be correlated to 
each of the variables. The quantitative data was analyzed using Pearson's Product- 
Moment Correlation with a bivariate analysis to determine the degree of significance 
to more objectively assess the correlation. 



Conclusions and Results 

The Change in Autonomy: The fiscal autonomy of the thirty SUNY • 
community colleges increased by twelve percentage points (from 80 percent in 1988, 
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to 92 percent in 1994), most significantly in the colleges' authority to name their own 
treasurer, to have their own bank accounts without sponsor intrusion, to retain interest 
on all college funds, to make their own investment decisions, and to transfer funds 
between budget categories. While the Plan A colleges gained the most autonomy, 
they remain, as of 1994, less autonomous than the Plan C colleges with critical issues 
yet to be resolved regarding authority to do their own purchasing, to maintain their 
own accounting and payroll systems, and to make unrestricted transfers between 
budget categories. 

Personnel autonomy also increased, but to a lesser extent than fiscal 
autonomy. With faculty and professional staff, significant increases occurred in the 
colleges' authority to promote faculty without sponsor approval, to offer fringe 
benefits different from the sponsor, and to add faculty and administrative lines after 
budget approval. Significant decreases, however, also occurred in the colleges' 
authority to bargain directly with the faculty. 

With civil service staff, other than the authority to add non-professional lines 
after budget approval, less than half of the colleges can act without sponsor approval 
to offer their own fringe benefits, to upgrade secretarial positions, to process 
grievances, to increase salaries, and to bargain directly with the civil service. Again, 
while the most significant increases occurred among the Plan A colleges, surprisingly, 
a minor decrease in personnel autonomy occurred among the Plan C colleges. 

In operational matters, the colleges' authority to use other than the county 
attorney continues to be a critical autonomy issue. Additionally, the intent of the 
legislation to establish a uniform state-wide fiscal operating system was not achieved, 
and is an unrealistic expectation under the current organizational structure of the 
SUNY community colleges. While the legislation ensured that all the colleges would 
receive their fimds in the same way, through a lump sum transfer from the sponsor, it 
did not ensure uniformity in the parameters through which those fimds could be 
expended. 

Factors and Strategies; Of the three clusters of variables examined, a 
significant correlation was found between autonomy and five institutional and 
contextual variables: the size; geographic location; level of resource dependence on 
the local sponsor; historical development of the college; and the county charter. 
Collectively, the higher autonomy colleges appear to be the smaller, rural colleges 
that are minimally supported by local sponsor fimds. These colleges also tend to have 
had a history of autonomy from the local sponsor, and tend to have been unrestricted 
by the county charter. This profile is consistent with the Resource Dependence model 
indicating that lower dependence on the environment, in this case lower financial 
dependence on the sponsor, translates into greater autonomy. 

In addition to the autonomy-related factors suggested in the literature, the 
qualitative data of this study suggest that autonomy is also significantly correlated to 
the local political and economic influences, and to the level of interpersonal 
communication and trust between the college and the sponsor. It is thought that as 
long as the sponsor has budgetary control over the college, the autonomy of the 
college and the nature of sponsor influence will vary given the economic times and 
the political philosophy of the individuals who make up the legislative body. While 
the community colleges are often viewed as "the best game in town", they have not 
successfully translated their benefits to the county into political capital. 
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Institutional Effectiveness 



Adaptation: According to these data, autonomy is not significantly correlated 
to curriculum innovation and vitality, nor does it significantly hamper the college in 
its mission to be adaptive and responsive to local community educational needs. 

Goal Attainment: Graduation Rates - Of the two cohorts examined, the 

analysis of the graduation rates showed a significant, positive correlation with 
autonomy for one cohort, and a strong but not statistically significant correlation for 
the other cohort. While these findings are inconclusive, they identify this as an 
important area for continued research. 

Resource acquisition - A strong negative correlation was found between 
autonomy and funding from the sponsor. The higher autonomy colleges receive a 
significantly lower proportion of their funding from the sponsor, relying instead on 
alternative revenue from charge backs for their out-of-county students. While the 
lower autonomy colleges receive a higher proportion of funding from the sponsor, 
they are subject to a greater degree of regulation, and they also experienced a 
significantly greater decrease in sponsor support when fimding patterns were analyzed 
over time, between 1988 and 1994. 

Integration: Autonomy is perceived to affect the salaries and qualification of 
the faculty and administrators with disruptive challenges to the colleges' authority in 
key personnel decisions. Autonomy is not perceived to be a significant influence on 
the academic climate of the college, nor on the way funds are allocated to the colleges' 
various academic units. 

Latency: A significant positive correlation is perceived to exist between 
autonomy and the motivational climate of the administrative domain of the 
institution, with concerns among the lower autonomy colleges about the level of 
bureaucracy, demands on administrators' time, and distractions from the academic 
work of the college. The correlation with the motivational climate of the instructional 
domain, and with teaching faculty specifically, is perceived to be minimal because of 
their limited interaction with the sponsor. Some concern was expressed, however, that 
the faculty may misunderstand the autonomy issues and view the college leadership as 
powerless. Additionally, the students are perceived to be unaffected by autonomy 
issues, primarily because the potential adverse effects of autonomy are mitigated by 
the professionalism and resourcefulness of the faculty and staff. The recurring theme 
throughout the data is the difficulty in measuring the relationship between autonomy 
and the effectiveness of the institution because, despite controls, the college personnel 
find ways to do what is necessary to provide a high quality education. 



Significance for Practice and Theory within Higher Education 

This study is particularly significant in light of the shared interest and 
responsibility of higher education and government to provide quality educational 
services under the stringent fiscal conditions that currently prevail. While building on 
the current limited empirical evidence about the autonomy of the community college 
sector of public higher education, the findings of this study support several 
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recommendations for practice and theory. 

For the practitioner, for example, future autonomy efforts would be better 
directed to focus on specific areas where additional flexibility is needed, coupled with 
the specific improvements that should result - improvements that legislators and the 
public can measure and understand. Future research would be well served to explore 
how increased autonomy has improved the effectiveness and productivity of the 
college, administratively and academically. 

For community colleges similar in structure to those of SUNY, the lack of 
uniformity under the Plan C legislation points to questions about the role of SUNY as 
the state coordinating agency. What is the role of the state coordinating agency - 
mediator, arbitrator, negotiator, advocate - in issues that are often as equally related to 
education as they are to local economic and political conditions? More basic, 
perhaps, is to question the practicality of the organizational structure of the SUNY 
community colleges that requires them to report to two bosses, one of which (the local 
sponsor) holds the power of a negotiated budget and wields control far beyond the 
extent of its financial support. Further study of the role of the state coordinating 
agency, and the organizational and funding structure of the community colleges is 
warranted if significant change is to occur. 

The political dimensions of the public college-sponsor relationship require 
that college leaders are equally adept working in the political and academic arenas. 
Further development of the political leadership of the SUNY community colleges, 
and of public higher education institutions in general, needs to continue along with 
further research on the political nature of the college-sponsor relationship. 

Understandably, complete autonomy, just as complete accountability, is 
unattainable and inappropriate in light of legitimate public interests about the funding 
and the role of public higher education. Continued efforts toward increased academic 
accountability are needed to strengthen the position of higher education in the 
continuing policy debates concerning the autonomy of public higher education 
institutions. 
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DEVELOPING A POSTSECONDARY EDUCATION TAXONOMY FOR 
INTER-INSTITUTIONAL GRADUATION RATE COMPARISONS 



Timothy A. Walsh 
Temple University 



Introduction 



Much of the research on postsecondary completions over the past fifty years has concentrated on 
student characteristics that influence retention, attrition and graduation (Pascarella and Terenzini, 
1991, Tinto,1987). The studies of institutional factors have been limited to only a few 
institutions, with large differences in graduation rates - primarily due to differences in definitions 
and methodology (Robb and others, 1988). In the 1980s public interest in college graduation 
rates grew, in part because of the concern over exploitation of football and basketball student 
athletes, but also by demands for higher education accountability. Within higher education, 
graduation rates were adopted as measures for assessment and Total Quality Management 
benchmarking. The Student Right-to-Know and Campus Security Act of 1990 provided a 
standard methodology for calculating and reporting graduation rates for almost all higher 
education institutions in the United States. With the growing interest in graduation rates, 
colleges are asking who are their peers for graduation rate comparisons. 

This study took the student variables associated with college graduation and matched them with 
publicly available institutional data to first develop a taxonomy for graduation rate comparisons. 
This taxonomy was then used to group colleges according to these factors. 

Methodology 

The literature identifies several student and institutional variables that are associated with college 
graduation. Successful completion of college has been linked to the demographic variables of 
gender, ethnicity, age and socio-economic status. Academic ability is often associated with high 
school class rank, standardized test score, and remediation. Several institutional characteristics 
are also associated with graduation. Institutional setting (city, suburb, small town or rural), 
number of other colleges in the area, region of the country, sector, mission, size, housing, 
student-faculty ratio, proportion of the E&G budget allocated to student services, selectivity, and 
full-time attendance. 

These variables associated with graduation were matched against publicly available data. The 
analysis used data for the 1991-92 academic year. The six-year graduation rate used the 1985 
cohort for each college. The Integrated Postsecondary Education Data System (IPEDS) provided 
considerable data. The Fall Enrollment Report provided headcount by academic level, full/part- 
time attendance, gender and ethnicity data. Total Educational & General (E&G) funds, student 
services expenditures and federal research revenue were available from the IPEDS Financial 
Report. The IPEDS Institutional Characteristics report provided data on sector, region and the 
number of other institutions in the area (using the first three digits of the zip code). The number 
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of full-time faculty came from the IPEDS Salaries, Tenure, and Fringe Benefits of Full-Time 
Instructional Faculty report. 

Average age of undergraduates, percent of undergraduates receiving financial aid (a proxy for 
socio-economic status), percent of freshman applicants admitted, average freshman combined 
SAT score, percent of freshmen from the top 10%/25% of their high school class, Carnegie 
classification and percentage of students taking only remedial courses were obtained from Minter 
Associates. Peterson's Guide was used to obtain the number of students living in residence halls. 
Barron's Guide provided the institutional setting (urban, suburban, small town or rural). ACT 
scores were converted to SAT scores based upon a College Board conversion table, 

The Carnegie classifications could not be used in subsequent analysis. However, the component 
parts (liberal arts/ comprehensive curriculum focus, doctoral degrees awarded and federal 
research dollars) could be substituted in the subsequent analysis. 

A total of 998 bachelors degree institutions had all the necessary data fields and could be used for 
subsequent analysis. 

ANALYSIS 

This study used factor analysis to identify the underlying structure of the data. These factors 
were then used in cluster analysis to group institutions with similar characteristics associated 
with graduation. It was hoped that cluster analysis would be a mathematically elegant solution to 
grouping colleges, rather than relying upon investigator judgement. 

While not required for factor analysis, zero order correlations were calculated. The primary 
reason for performing this analysis was to refine two variables. The percent of freshmen in the 
top ten and twenty-five percent of their high school class obviously measure the same thing. The 
student-faculty ratio could be calculated several ways. Ideally the FTE undergraduate students to 
FTE faculty who teach undergraduates would be used. However that data were not available. 
Instead, three variations of student-faculty ratio were calculated - total students to total full-time 
faculty, total undergraduates to total full-time faculty, and FTE undergraduates to full-time 
faculty. 

The percentage of freshmen in the top 25% of their high school class was dropped from further 
analysis since the top ten percent proportion had a higher correlation with graduation rate (.6355 
vs. .6094). Similarly, FTE student and total student-faculty ratios were dropped because the 
undergraduate student-faculty ratio had a higher correlation with graduation rate (-.3282 
compared to -.2983 and -.2690 respectively). The other variables that failed to be significantly 
correlated with graduation rate were also dropped for simplicity of further analysis. Total 
headcount and total undergraduates failed to be significantly correlated with six year graduation 
rate. Proportion of the undergraduates who are male was also dropped. Setting (urban/non- 
urban) was not significant, nor was the proportion of the E&G budget allocated for student 
services. 



FACTOR ANALYSIS 

The remaining variables were used in an'SPSS factor analysis (Table 1). Five factors emerged 
with an eigenvalue greater than 1 .0. These five factors explain 67.7% of the variance in 
graduation rate. 

The presence of non-traditional students explained 24.9% of the variance. The variables that 
contribute most to this factor are the average age of undergraduates, percent of undergraduates 
attending part-time, percent of imdergraduates living in housing, the number of other colleges in 
the area, and the undergraduate student-faculty ratio. 

Student academic ability explained an additional 17.4% of the graduation rate variance. The 
variables that contribute most to this factor are the average SAT score, percent of applicants 
admitted, percent of freshmen in the top ten percent of their high school class, percent of 
imdergraduates receiving financial aid, comprehensive/ liberal arts curriculum focus and the 
number of other colleges in the area. 

The third factor was institutional mission, which explained an additional 9.1%. The variables 
that contributed most to this factor were federal research dollars, doctoral degrees awarded and 
liberal arts/ comprehensive curriculum focus. 

The institutional sector is the fourth factor (8.7% of the graduation rate variance). It is primarily 
defined by the percent of undergraduates receiving financial aid, public/ private sector, and the 
number of colleges in the area. 

The fifth factor is the presence of disadvantaged students (explaining 7.6% of the graduation rate 
variance). Average freshman SAT score, percent of freshman applicants admitted, percent 
minority students, and percent of undergraduate students taking only remedial courses are the 
variables that primarily define this factor. 

CLUSTER ANALYSIS 

Factor scores were calculated for each of the 998 bachelors degree granting colleges. The factor 
scores were standardized to compensate for differences in scaling. There are a variety of ways to 
calculate clusters. Distances between clusters can be calculated by measuring the squared 
Euclidean distance between the nearest two observations, or the furthest, or the average distance. 
This analysis used the SPSS Cluster function using the nearest neighbor to define the joining of 
groups. However, the 998 by 998 correlation matrix necessary for cluster analysis could not be 
handled by the mainframe computer. 

While the SPSS Cluster function would not work, another SPSS program called Quick Cluster 
would. With Quick Cluster the researcher specifies the number of clusters desired. Solutions for 
ten, twenty, thirty, forty, fifty and one hundred clusters were calculated. After investigating the 
groupings, it appeared that the optimal solution was somewhere near thirty clusters. Twenty-five 
to thirty-five cluster solutions were then calculated. The thirty cluster solution appears to be the 
best grouping of colleges (Table 2). 
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In the process of analyzing the cluster groupings it became evident that a purely mathematical 
solution of which colleges should be grouped as peers is not possible. Peer cluster determination 
is still as much an art as a science. The membership in each cluster may still be more dependent 
upon the purpose for the analysis. 

CONCLUSION 

The primary benefit of this analysis was the identification of the factors associated with 
graduation. The presence of non-traditional students, academic ability, mission, sector and 
proportion of disadvantaged students explain about 68% of the variance in graduation rates. It is 
possible that with the growing availability of data that additional variables that contribute to 
graduation will be obtainable. 

This study originated with the hope that there could be a niathematically elegant solution to the 
age old question of peer institution identification. The current cluster analysis techniques still 
require considerable judgement on the part of the researcher. 

The clusters that were formed still had considerable variance in graduation rate (up to 45%), even 
though the institutions and students were similar. There are other research initiatives that are 
looking at outliers (Krock et al.,1995) or looking at the reasons for differences between expected 
and actual graduation rates (Astin,1993). These initiatives may identify the reasons why 
supposedly similar institutions have disparate graduation rates 
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Table 1: FACTOR COEFFICIENTS FROM FACTOR ANALYSIS 
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Table 2: 



AVERAGE FACTOR SCORES FOR THE THIRTY CLUSTER SOLUTION 



Cluster 

Cluster 

Cluster 

Cluster 

Cluster 

Cluster 

Cluster 

Cluster 

Cluster 

Cluster 

Cluster 

Cluster 

Cluster 

Cluster 

Cluster 

Cluster 
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Cluster 

Cluster 

Cluster 

Cluster 

Cluster 

Cluster 

Cluster 

Cluster 

Cluster 

Cluster 

Cluster 

Cluster 

Cluster 

Factor 

Factor 

Factor 

Factor 

Factor 



o 






Number of 
Cases 


Non- 

Traditional 
Students : 
FACTOR 1 


Student 
Academic 
Ability : 
FACTOR 2 


1 


31 


1.10762 


0.41186 


2 


4 


0 . 00977 


2 . 92457 


3 


136 


-0.75031 


-0.88422 


4 


17 


-0.45628 


0.70025 


5 


8 


-0.78285 


3 .22693 


6 


4 


-0.23314 


2 .09444 


7 


2 


-1.61604 


-0.84529 


8 


3 


-1.27552 


0.87339 


9 


1 


-0.65823 


-0 . 98646 


10 


42 


-0.08525 


0.02316 


11 


2 


-0.24220 


3 .51852 


12 


9 


2 .77962 


-0.18461 


13 


58 


1.29529 


-0 . 63977 


14 


25 


-0.93038 


-1 . 13646 


15 


2 


1.67022 


-0.06900 


16 


4 


-0 . 96706 


1.96669 


17 


47 


-0.93319 


1. 98992 


18 


9 


-0 . 18754 


0.21675 


19 


1 


1.08060 


1.89369 


20 


21 


1.56532 


0.50909 


21 


75 


1.27626 


0.36817 


22 


3 


-0.57063 


1 . 18760 


23 


153 


-0.78400 


0.39020 


24 


1 


-1.01938 


-0.09069 


25 


11 


-1.03451 


2 .45080 


26 


15 


2.88376 


0.48250 


27 


133 


0.21758 


-0.44677 


28 


12 


-0.34825 


1.61235 


29 


147 


0.00340 


-0.48425 


30 


22 


-0.20775 


-0 . 87793 



Disadvantaged 



Mission : 
FACTOR 3 


Sector : 
FACTOR 4 


Students : 
FACTOR 5 


0 . 15262 


2.02104 


0.47755 


0 .41190 


2.23967 


0.90868 


-0.01137 


0.39693 


-0.25751 


1.72462 


0.73259 


-0 . 16547 


-1.05344 


0.91304 


0.53203 


3 .72090 


1 .95731 


-0.10363 


-1.10799 


-0.01035 


9.73234 


-1.37415 


-0.51113 


2.60919 


0.02125 


-0.65124 


4.88222 


1.55751 


-1.27213 


-0.35855 


4 . 93834 


3 . 57116 


0.55393 


-0 . 70776 


1 . 58141 


-1.27258 


-0 . 03381 


0.84608 


-0.41286 


0.20831 


0.88120 


2 . 82017 


0.34356 


1.38252 


. 3 .54589 


5.87984 


-0.18600 


0.04568 


-0 . 96140 


-0.01410 


-0.16430 


4 . 55721 


-0. 93957 


-0 . 82378 


5.52550 


2 .40628 


-0.73123 


0 . 10763 


-0 . 11709 


1 . 11398 


-0 .34826 


-1.27931 


0.00935 


-0 .57095 


1.59720 


3 . 53665 


-0.79792 


0 . 18900 


-0.58604 


-1.76482 


-1.70134 


8.05987 


2 . 14791 


-0 . 24664 


0.13391 


-0.24396 


0 . 53863 


-0.01986 


-0 . 09063 


0 . 56508 


-0.21931 


0.47064 


-1.49501 


0.73823 


0 .01715 


-1.20050 


-0.08829 


0.09381 


-0.33208 


2 .22151 



- Non- tradit ional Students [non- traditional is positive] 

- Student Academic Ability [high academic ability is positive] 

- Mission: Research/ Liberal Arts Focus [research focus is positive] 

- Public/Private Sector [private sector is positive] 

5 - Disadvantaged Students [high proportion of minority students is positive] 
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INCREASING THE UTILITY 

OF THE COLLEGE BOARD STANDARD VALIDITY STUDY 
AN MS-DOS QBASIC PROGRAM 

FOR PREDICTING AN APPLICANT’S COLLEGE PERFORMANCE* 

Jishen Zhao 

Statistician/ Analyst of Institutional Research 
The College of Saint Rose 

Introduction 

Multiple linear regression analysis has been widely used in higher education 
enrollment management. Based on an optimal linear combination of an applicant’s 
SAT scores, high school GPA, and high school rank, this statistical technique 
provides college admissions officials with a prediction model that generally forecasts 
a prospective student’s college performance with decent accuracy. 

As the most popular option of the College Board Validity Study Service, the 
Standard Validity Study prepares freshman GPA prediction models tailored to the 
specific needs of individual institutions. While doing so, it also measures the 
accuracy of the predictors and estimates the magnitude of the prediction error (The 
College Board, 1988). Conceptually, a typical report from the Standard Validity 
Study can be viewed as a combination of two parts: model presentation and model 
application. Describing and evaluating the prediction model and its supporting data 
set, the model presentation component contains (a) frequency distributions for each 
variable; (b) a summary of descriptive statistics for each variable, and correlational 
statistics between the predictor(s) and the criterion; (c) an analysis of predictor- 
criterion relationships, including multiple correlations, standard errors of estimate, 
and regression coefficients; (d) an experience table of predicted and obtained grades, 
and (e) correlation matrices indicating correlations among all variables. Ameliorating 
the application of the prediction model in recruiting potential students, the model 
application component involves (a) a computational aid to approximate the predicted 
grades, and (b) an expectancy table to assess an applicant’s probability of obtaining 
certain grades. Although the College Board has prepared very readable 
documentation helping the consumer understand the prediction model presented in its 
study report, the lookup tables it supplies for model application compromises the 
utility of the study for two reasons. First, admissions officials very often may not 
have the time to consult a prediction model in their daily operation. However, when 
they turn their attention to this enrollment management aid, they are probably looking 
for the most accurate information possible to facilitate their decision-making process. 



* The author wishes to thank Dr. Donald Dean, Mary Grondahl, Dr. Rusan Chen, Nancy Ervin, and 
Felicia Lyu for their help and support in developing FRGPA. 
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A lookup table such as the computational aid that contains the mid-points of some 
twenty intervals rather than all possible points apparently will not meet their 
information needs. Second, just as a college student nowadays will use a calculator 
rather than a slide rule to solve a mathematical problem, it is probably not efficient in 
this computer age to use a lookup table at all when virtually every college campus has 
realized its office automation. 

In order to increase the utility of the College Board Standard Validity Study, the 
model application component of the study report was computerized to help 
admissions officials predict an applicant’s college performance. MS-DOS QBasic, a 
programming language that resides in certain Microsoft operating systems, was 
chosen for this purpose because of its common accessibility and user-friendliness. In 
addition to the prediction aids, a decision-theoretic model was included in the 
computer program to quantify the utilities of admissions decisions. _ , 

Program Description 

Freshman GPA Prediction and Extensive-Forms Analysis 

FRGPA, an MS-DOS QBasic program, was developed based on the College Board 
Validity Study for The College of Saint Rose. In place of the computational aid in the 
study, this computer program directly estimates a prospective student=s freshman 
grades {GPA) based on his/her high school average (HSAV), SAT scores {SAT), and 
high school rank {HSRANK), using the following well-tested multiple linear 
regression equation from the study report: 

GPA = Ro + B\HSA V + B^SAT + B^HSRANK. 

At the same time, FRGPA supplies the relevant information from the expectancy 
table by computing the applicant's probability of earning a certain GPA or higher. 
Assuming that the obtained GPAs ( GPA^ s) of the applicant with a predicted GPA of 
g distribute normally with a mean of g and a standard deviation equal to the standard 
error of estimate ) of the regression equation, the applicant’s probability of 
obtaining a GPA^ or higher can be found by determining the area under the standard 

normal curve for the z score of — — (The College Board, 1988). 

^GP/l 
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Starting from the calculated probability {p\X)_ that the applicant will gain the 
cutoff GPA score (X), FRGPA further estimates the expected utility of a select 
decision ( s (u )^ ) as well as the expected utility of a reject decision ( s (w)^ ), using the 
following extensive-forms analysis model (Crocker & Algina, 1 986): 

£ («).v = {p\^) + W/p {.q £ (w), =Uj,{p\X) + u,„ (q\X), 

where 

= utility of a true positive event, Uj^ = utility of a false positive event, 

Uj^ = utility of a false, negative event, = utility of a true negative event, 

p \X= probability of success, and q |X = probability of failure (i.e., \ - p\X) 

Novick and Lindley (1978) discussed the procedures to compute utilities of various 
events. For FRGPA, a simplified approach was adopted by utilizing the following set 
of numerical values from Crocker and Algina (1986, p. 279): 

u,p = 1.00, w,„= 1.00, Uj-„ = .25, and = .75. 

The FRGPA User may want to compare the two expected utility values when they are 
available. It is advised that the decision alternative with a greater expected utility 
should be -a rational choice. 

Operational Procedure 

1. Start MS-DOS QBasic: 

• Move the mouse until the arrow points to the Accessories program group 
icon in Microsoft Windows, and double click. 

• Move the pointer to MS-DOS QBasic program item icon in the 
Accessories window and double click. 

2. Start FRGPA: 

• Press the Esc key to clear the dialog box after the welcome message of 
MS-DOS Qbasic appears. 

• Move the pointer to File in the menu bar and click to open the. pull-down 
menu. Move the pointer to the Open command, and click to activate its 
dialog box. 

• Move the pointer to [-A-] in the Dirs/Drives section of the Open dialog 
box and click. to change the drive (Assuming FRGPA is kept on a floppy 
diskette in A drive) . 



• Move the pointer to FRGPA.BAS in the Files section of the Open dialog 
box and click to select the code. 

• Move the pointer to < OK > at the bottom of the Open dialog box to start 
the application. 

S.RunFRGPA: 

• Move the pointer to Run in the menu bar after the FRGPA code appears 
on the screen, and click to open the pull-down menu. Move the pointer to 
the Start command, and click to activate its dialog box. 

• Enter the following applicant information as prompted when the FRGPA 
identification screen shows up; Name, Social Security Number (in the 
format of ###-##-####), High School Average, High School Perfect 
Average, High School Rank, High School Class Size, SAT-Verbal, and 
SAT-Math. Press the Enter key after each piece of information is typed 
in. 

4. Exit or restart: 

• Press any key to return to the FRGPA code screen after the one-page, 
output rolls out of the printer. 

• Begin another run of the program by activating the Run menu again, or 
quit the program and return to the Accessories window by clicking Exit in 
the File menu. 

It should be mentioned that the above procedure can also be completed on the 
keyboard alone. Besides, MS-DOS QBasic is not a windows program, and FRGPA 
can be accessed in a DOS environment as well. 

Technical Notes 

Two technical issues involved in developing FRGPA need to be addressed briefly: 

1 . Prediction model The College Board Validity Study Service proposed five 
prediction models for our college based on five consecutive years= data. Since ours is 
a small comprehensive type II institution with only an average of 1 60 valid cases for 
an annual undergraduate enrollment analysis, the five prediction models are not 
totally stable in a statistical sense. In an effort to increase the sample size underlying 
the prediction model, a multivariate analysis of variance (MANOVA) was conducted 
to identify those classes of freshmen who were statistically more homogeneous in 
terms of their mean high school average, high school rank, and SAT scores. As a 
result, a total of 463 freshmen excluding outliers in three eligible classes were 
selected to form a larger sample, and a new prediction model was built accordingly. 



2. Normal distribution table In order to computerize the expectancy table from 
the Standard Validity Study report, a normal distribution table must be included in the 
computer program that indicates the areas under the standard normal curve for z 
scores between .01 and 3.00. As MS-DOS QBasic carries no function that serves this 
purpose, a SAS/IML program was written to generate normal distribution value 
statements in an MS-DOS QBasic format that were subsequently copied into the 
program. 

Program Availability 

FRGPA runs on any decent IBM-compatible PCs equipped with the MS-DOS 
QBasic program. The interested colleague should send a 3.5-inch blank IBM- 
formatted diskette along with a pre-stamped, self-addressed diskette mailer to the 
following address: 

Jishen Zhao 

Satistician/ Analyst of Institutional Research 
The College of Saint Rose 
432 Western Avenue 
Albany, NY 12203. 

To receive a customized program for your institution, an information sheet should 
also be sent that specifies (a) your institution’s prediction model, (b) its standard 
error of estimate that usually appears in the Analysis of Predictor-criterion 
Relationships section of a Standard Validity Study report, (c) desired GPA points for 
the expectancy table, and (d) the cutoff score for the extensive-forms analysis. It is 
highly recommended that an FRGPA user inform the author of the status of the 
program usage within a month after FRGPA is received. Any comments and 
suggestions regarding the program will be greatly appreciated. 

Limitations of FRGPA and Prediction Models in General 

As a tentative attempt to computerize the model application component of the 
College Board Standard Validity Study report, FRGPA is still limited in the number 
of prediction aids it carries. An extensive-forms analysis model is included, yet the 
assignment of utility values remains largely arbitrary. FRGPA should be used with 
caution for various reasons. The most important one the user should be aware of is 
the inherent limitations of multiple linear regression analysis. The literature is 
abundant in terms of prediction errors resulting from such factors as range restrictions 
and heterogeneous subsamples. The FRGPA user is referred to Zubrow (1989) for a 
very relevant discussion. 
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NEAIR 23rd Annual Conference Program 


Saturday. November 16 


1 :00 - 6:00 p.m. 


Conference registration in the Nassau Foyer 


2:00 - 6:00 p.m. 


Facilitating Change, Part 1 


Michael Matier 
Associate Director of 
Institutional Planning & Research 

Peter Hurst, Senior Research and 
Planning Associate 


This experiential workshop is for institutional researchers who 
wish to expand their knowledge base and abilities as change 
agents. We will explore group dynamics, the nature of leadership 
in group situations, and the facilitation of individual and 
organizational change. Warning: this work is fun! 

Workshop 


Cornell University 




CANCELLED 




2:00 - 5:00 p.m. 

Nassau A/B 

J. Fredericks Volkwein 
Director of Institutional Research 
& Assoc. Professor Educational 
Administration & Policy Studies 

SUNY-Albany 


Managing a Program of Outcomes Assessment, Part 1 

This workshop acquaints institutional researchers with the tools 
and processes for assessing campus educational outcomes. 
Participants will review the multiple purposes and uses of 
assessment, and will learn about various methodologies and 
instruments that are available and appropriate. Participants 
should bring copies of assessment plans and reports from their 
own institutions where possible. 




Workshop 


2:00 - 5:00 p.m. 
Colonial/University 


Newcomers to Institutional Research: Strategies 
for Effective Institutional Research, Part 1 


Karen Bauer 
Assistant Director of 
Institutional Research and 
Planning 

University of Delaware 


This workshop is designed for new practitioners who engage in IR 
activities. Usino the AIR monoaraoh. Strategies for the Practice 
of Institutional Research, the workshop addresses key 
components of IR including defining critical issues for institutional 
research, identifying sources of data, developing factbooks and 
other reports, and conducting effective survey research for 
assessment and evaluation. The main focus is a presentation of 
general concepts and practical strategies for the implementation 
or continued development of effective IR at many schools, 
regardless of size or type. 




Workshop 



o 
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Saturday, November 16 continued 


2:00 - 5:00 p.m. 


Stress Reduction for Frayed Researchers 


Carey R. Trevisan 
Director of Admissions and 
Records 

Ocean County College 


The president expects you to drop everything to prepare for an 
emergency board meeting. You promised your son you'd be at 
the recital. A client has called three times to complain that the 
analysis doesn't support his prejudices -- so it must be wrong. 
Your C: drive is acting funny and your best analyst just gave 
notice. 


CANCELLED 


Stress -- it's part of all our lives. If we know how to manage it, 
our lives will be happier. This workshop will help you recognize 
stress in its daily forms: physical, emotional, and psychological, 
and will cover methods of controlling it in your day-to-day 
existence. 




Workshop 


6:00 p.m. 


Reception in the Greenhouse Restaurant 
Sponsored by Peterson's. 


Sunday, November 17 


8:00 a.m. - 5:00 p.m. 


Conference Registration continues in the Nassau Foyer 
Vendor exhibits and demonstrations throughout the afternoon 


8:30 a.m. - noon 


Facilitating Change, Part 2 


Michael Matier and Peter Hurst 
Cornell University 

CANCELLED 


Continuation; Part 1 is a pre-requisite. 
Workshop 


9:00 a.m. - noon 
Nassau A/B 

J. Fredericks Volkwein 

SUNY-Albany 


Outcomes Assessment, Part 2 

Continuation; Part 1 is a pre-requisite. 
Workshop 


9:00 a.m. - noon 
Colonial/University 

Karen Bauer 

University of Delaware 


Newcomers to Institutional Research, Part 2 

Continuation; Part 1 is a pre-requisite. 
Workshop 





206 



Sunday, November 17 continued 




9:00 a.m. - noon 


Statistics for Institutional Research, Part 1 


Library 


In Part 1 the very basic ideas in statistics will be covered in a 


Mary Ann Coughlin 


way useful as an introduction or as a refresher to statistics. 


Professor of Research & 


Descriptive statistics, sampling and probability theory as well as 


Statistics 


the inferential methods of chi square, t-test, and Pearson's r will 
be covered. May be taken with or without the follow-up 


Springfield College 


advanced workshop. 
Workshop 


Noon - 1 ;00 p.m. 


Buffet lunch in Ship's Room for Sunday workshop participants. 
Tickets required. 


1 :00 - 3:00 p.m. 
Colonial/University 


Catholic Colleges and Universities 

Representatives of Catholic colleges and universities are invited 


Linda Winkler 


to share experiences and common concerns and to plan activities 


Director of Institutional Research 


of mutual benefit. 


Mount Saint Mary's College 
John Biter 

Director of Institutional Research 


Special Interest Group 


St. Bonaventure University 




1 :00 - 4:00 p.m. 
Nassau A/B 


Questionnaire Survey Research: What Works 

This workshop serves as an introduction to planning and 


Linda Suskie 


designing an effective paper-and-pencil survey. Topics to be 


Assistant to the President for 


discussed include planning the survey, the pros and cons of 


Special Projects 


various question formats, and ensuring adequate validity and 
reliability and a high response rate. Visual presentations will be 


Millersville University 


supplemented by group exercises and discussion. 
Workshop 


1:00 - 4:00 p.m. 
Library 


Advanced Statistics for Institutional Research 

This workshop will deal with advanced issues in inferential 


Mary Ann Coughlin 


statistics. Topics such as Analysis of Variance, Factor Analysis, 
Multivariate Regression, and Logit/Probit models will be covered 


Springfield College 


and contrasted with other statistical tools and techniques. A 
case study approach will be used illustrating applications of these 
statistical techniques in institutional research. SPSS running on a 
PC will be used for this workshop. Open to those who have 
completed the introductory workshop on Sunday morning, or who 
have an equivalent background. 

Workshop 
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Sunday, November 17 continued 




1 :00 - 4:00 p.m. 
Princeton 


Web Basics for Institutional Researchers 

This workshop will be an introduction to the basics of finding 


James C. Fergerson 


resources on the web and putting information on the web for 


Director of Institutional Research 


institutional researchers. It is intended to give relative beginners 
some of the practical and technical background needed to locate 


Bates College 


institutional research information on the web and to set up a 
basic web site. It will include basic introductions to Hypertext 
Markup Language (HTML) and to popular HTML editing tools. 
The "do's and don'ts" of setting up a public web site will be 
discussed. 




Workshop 


4:00 - 5:1 5 p.m. 


Taking Survey Research to the Next Step: Empowering Social 
Change 

George Gallup, Jr., President, Gallup, Inc. 

Opening Plenary Session in the Prince William Ballroom 


5:15- 7:00 p.m. 


President's Reception in Senior Room and Assembly Area 
Sponsored by Scanning Products, Inc. 


5:1 5 - 6:00 p.m. 


Mentor's Meeting in Ship's Room 


7:00 - 10:00 p.m. 


Banquet and Entertainment in Prince William Ballroom 

Marc Abrahams, Editor, "The Annals of Improbable 
Research" (AIR!) and father of the Ig Nobel Prize 

Advances in Improbable Research 

A seminar/slide show presenting outstandingly improbable 
research, a surprising amount of it genuine. The talk 
includes the highlights from the Ig Nobel Prize Ceremonies 
held annually at Harvard. Topics span the full range 
of sciences and beyond, including: The Taxonomy of 
Barney; a Spectrographic Comparison of Apples and 
Oranges; Tabletop Fusion and Feline Reactions to 
Bearded Men. Heckling is encouraged, as are lab coats 

and other 

appropriate and inappropriate regalia. 
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Monday, November 18 


8:00 - Noon 


Conference Registration continues in the Nassau Foyer 


7:45 - 8:45 a.m. 


Breakfast in the Ballroom; Concurrent At-Your-Service 
Sessions and Special Interest Groups 


7:30 - 8:45 a.m. 


COFHE Breakfast 


Princeton University Faculty Club 


A chance for COFHE schools to share ideas and concerns. 


Kathleen Kern Bowman 
Research Associate 


Special Interest Group 


COFHE 

**************************** 


Two-Year Colleges 


8:00 - 8:45 a.m. 
Table 2 


Problems, concerns and issues for the IR function in two-year 
colleges will be shared in an informal setting. Concerns about 


Alan J. Sturtz 


the impending IPEDS GRS-2 survey will be a key discussion 


Director of Institutional Research, 
Planning and Assessment 


topic. 




Special Interest Group 


Gateway Community-Technical 
College 

***************************** 




8:00 - 8:45 a.m. 


SUNY AIRPO 


Table 3 


Agenda includes marketing strategies that don't hurt one 


Marlene Arno 


another; SUNY Cental assistance and the Federal Graduate 


Director of Institutional Research 


Rate Survey. 


Erie County Community College 
***************************** 


Special Interest Group 


8:00 - 8:45 a.m. 
Table 4 


NJAIR 


Eleanor Swanson 


Topics to be covered include discussion of the steering 


Director of Institutional Research 


committee's proposal for formalizing the organization of NJAIR 
and planning for the third annual spring meeting of NJAIR. 


Monmouth University 
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Monday, November 18 continued 


9:00 - 9:45 a.m. 


Developing a Survey Program: Insights from the American 


Princeton 


University Experience 


Karen Froslid 


The presenters discuss development of a survey program. Topics 


Research Analyst 


to be covered: Deciding what surveys to include in a program, 
designing survey instruments, combining several surveys to form 


Elizabeth Sibolski 


a program, and contributing to policy outcomes. An open 


Director, University Planning & 


exchange of ideas will provide an opportunity to learn from each 


Research 


other's experience. 


American University 


Workshare 


9:00 - 9:45 


Accessing National Science Foundation Data Over the Internet 


Nassau A/B 


This session will help planners and institutional researchers take 


James W. Firnberg 


advantage of the vast amount of National Science Foundation 


President 


data and reports available over the Internet. Institutionally 
specific as well as various sub-totals of data are available. The 


Our Lady of the Lake College 


presentation will also demonstrate projects in the planning or pilot 
testing stages. 


George Nozicka 
President 


AYS Technical Topic 


Quantum Research Corp. 




9:00 - 9:45 a.m. 


Graphic Design Basics for the Institutional Research Office 


Senior 


The means by which most institutional research findings are 


Kathleen Keenan 


communicated is still the printed page. While software has 


Director of Institutional Research 


increased our ability to produce attractive, professional looking 
reports incorporating graphic elements, it cannot make design 


Massachusetts College of Art 


decisions for us. This workshare will focus on the visual 
dimensions of the documents we produce. Participants are 
invited to bring examples to share - reports, questionnaires, 
brochures, etc. - which they think represent either successful or 
flawed graphic design strategies for IR documents. 




Workshare 
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Monday, November 18 continued 


9:00 - 9:45 a.m. 


Fueling an Executive Information System 


Colonial/University 


We will provide an overview and details of how institutional 


Barbara Sadowski 


research at Marywood College creates and maintains documents 


Director of Planning and 


and reports for an EIS. Information on how mixed clients access 


Institutional Research 


these reports, the types of information shared, security concerns 
and legal issues concerning information storage will be shared. 


Marie Huester 
Charlotte Woodward 


Workshare 


Marywood College 




9:00 - 9:45 a.m. 


New to IPEDS: Graduation Rate Survey 


Ships Room 


This presentation will address the anticipated changes in the 


Susan G. Broyles 


IPEDS data collection for 1996 and 1997 for the Graduation Rate 


IPEDS Project Director 


Survey (GRS). NCES expects to pilot test the new survey this 
fall and implementation is planned for 1997. NCES will report on 


Roslyn A. Korb 


the development of the survey instruments, the current status of 


Program Director 


the pilot study, and review all versions of the forms. 


National Center for Education 


Audience participation is encouraged to discuss the institutional 


Statistics 


perspective, especially how the new forms will be dealt with at 
the institutions. 


NEW SESSION 




9:45 - 10:00 a.m. 


Break in the Nassau Foyer 




NEAIR is grateful for the support of its vendors. Be 
sure to visit their exhibits to learn about the latest 
products to enhance your office productivity and 
effectiveness. 
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Monday, November 18 continued 




10:00 a. m. - noon 


Institutional Research to Promote Institutional 


Princeton 


Effectiveness 


Charles Abel 


Nominal Group Technique: What Can We Do to Improve 


Dean of Graduate Studies & 


Our Graduate Programs? 


Director of IR, Planning & 


Assessment 


The Nominal Group Technique was utilized to identify the 


Brenda Bailey 


problems, strengths, visions and needs of graduate programs. 


Participants including graduate program heads, graduate studies 


Research Specialist 


staff, and a facilitator generated a rank-ordered list of ten 


Edinboro University of 


priorities for improving graduate programs. Ultimately the 


process led to the establishment of a continuous improvement 


Pennsylvania 


plan. 


ABOVE PAPER CANCELLED 


Research Paper 


10:00 - 10:30 a. m. 




Anne Marie Delaney 


The Institutional Researcher as Program Evaluator: Influencing 


Director of Program Research 


the Development of Innovative Programs 


Boston College 


Using the evaluation of a university based, three-year inter- 
professional program as a case study, this paper demonstrates 
how institutional researchers can expand their role to include that 
of program evaluator and thereby influence the development of 
innovative academic and professional degree programs. The 
paper presents the models of evaluation, research design, 
implementation strategies, and qualitative and quantitative 
analysis techniques utilized in this evaluation and relevant to the 




evaluation of other programs. 


10:40 - 1 1:10 a. m. 


Research Paper 


Dawn Geronimo Terkla 


Weaving Institutional Research Into the Fabric of TQM 


Director of Institutional Research 


Kelli Armstrong 


The process of Total Quality Management involves a research 


component, or a point where a quality improvement team must 


Research Analyst 


examine prior practices and current status before recommending 


Tufts University 


change. This paper examines the impact of improvement 


techniques like TQM on the resources of an institutional research 
staff and the approach that has been adopted at a Research 1 
University. 

Research Paper 

Moderator: Stuart Rich, Director of Institutional Research, 
Georgetown University 



Monday, November 18 continued 




10:00 a.m. - noon 

Nassau A/B 


Studies in Autonomy and Effectiveness 


J. Fredericks Volkwein, 


The Dynamics of State Regulation and Administrative Flexibility 


Director of Institutional Research 


This is a four-part study, supported by NEAIR, that investigates 
the dynamics of state regulation and administrative flexibility. 


Shaukat M. Malik 


The study first measures the academic, personnel and financial 


Lu Chen 


dimensions of state regulation, as well as the changes that have 
occurred since 1983. Second, the study examines the 


SUNY-Albany 


relationship between regulation and various state characteristics, 
and tests the hypothesis that state characteristics have an effect 
on the level of flexibility granted to university campuses. Third, 
the research analyzes the relationship between regulation and 
campus characteristics, and tests the hypothesis that 
administrative and academic controls are stimulated by campus 
characteristics or behavior. Fourth, the research analyzes the 
relationship between state regulation and the levels of 
satisfaction among those campus managers who theoretically 
might be impacted by regulation. 

Research Paper 


Anita Voogt 


Correlates of Autonomy and Institutional Effectiveness in NYS 
Community Colleges 


Dutchess Community College 


A two-part study of the change in autonomy of the community 


J. Fredericks Volkwein 


colleges of the State University of New York as a result of recent 
legislation, and the perceived relationship between autonomy and 


SUNY-Albany 


selected measures of institutional effectiveness. 

Research Paper 

Moderator: Kent W. Smith, Institutional Researcher, Trinity 
College 

This work is supported in part by the NEAIR Research Grant 
1 Program. 
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Monday, November 18 continued 



10:00 a.m. - noon 
Senior 



Historical Studies to Inform Public Opinion 



10:00 - 10:30 a.m. 

Michael J. Dooris 
Director of Academic 
Assessment 

O. Richard Bundy, III 
Associate Director, Telefund 

The Pennsylvania State 
University 



What Price a Nittany Lion? Tuition and Fees at Penn State from 
1859 to 1995 

How do changing tuition and fees relate to inflationary and 
deflationary trends, faculty salaries, or the salaries of college 
graduates? How do recent developments compare to long-term 
historical trends? This unusual paper addresses these and similar 
questions for one university, Penn State, from its founding in 
1 859 to the 1 990s. 

Research Paper 



10:40 - 1 1:10 a.m. 

John Kraus 

Director of Institutional Research 

Antonietta Taylor 
Assistant Director 

The University of New 
Hampshire 



Graduation Rates at the University of New Hampshire - An 
Historical Perspective 

Graduation rates are a "hot button" issue. Is higher education 
doing as good a job of producing graduates as previously? An 
analysis of records from the 1920s and 1930s enables 
comparison of UNH rates then and now. Conclusions and 
implications are outlined. 

Research Paper 



11:20 - 11:50 a.m. 
C. Anthony Broh 
Registrar 

Princeton University 



The Race is On 

Many people are surprised to learn that the methodology for 
classifying people by race and ethnicity has changed with every 
decennial census since 1790. The 2000 census will likely 
continue this unbroken string of continuing change with 
intervening consequences for several government reporting 
requirements -- including IPEDS. While the changes at first 
glance appear rather small, together they describe the way 
researchers have thought about race and ethnicity since the 
formation of the nation. 



Workshare 



Moderator: Timothy A. Walsh, Director of Student Information, 
Temple University 
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Monday, November 1 8 continued 


10:00 a. m. - noon 

Colonial/University 


institutional Research to Anchor institutional Policy 


10:00-10:30 
Alan J. Sturtz 
Director of Institutional 
Research, Planning & 
Assessment 

Gateway Community-Technical 
College 


Placement Test Scores and Student Persistence: Institutional 
Considerations for Ability to Benefit 

This study tracks all new students who entered Gateway 
Community-Technical College since fall 1987 who were 
recommended for and enrolled in developmental English as the 
result of a standardized placement test. The focal hypothesis is 
that, with due consideration to "Ability to Benefit," students who 
test below a given level may not receive satisfactory grades in 
the course and may leave the college. This analysis can provide 
vital decision support for academic advising and course 
development, and scheduling and institutional policy formulation. 


10:40 - 11:10 a.m. 

Keith J. Guerin 

Director of Institutional Research 
& Planning 

County College of Morris 


Research Paper 

Measuring the Effects of Basic Skills Policy Changes: A 
Multivariate Comparison of Apples and Oranges 

In 1994 the New Jersey Department of Higher Education was 
abolished. The new governor granted colleges greater autonomy 
in establishing policies for remedial course requirements. This 
paper examines the effects of eliminating objective tests for 
placement, and the replacement of four remedial courses with 
one. Retention rates, GPAs, and performance in specific college 
credit courses are examined for three first-time cohorts to assess 
the outcome of these changes. 


1 1:20 - 1 1:50 a.m. 

Stephen W. Thorpe 

Director of Institutional Research 


Research Paper 

IR Influence on Marketing and Pricing Policies: First-time 
Graduate Student Inquiries - Why Don't They Apply? 


La Salle University 


Institutions of higher education are expanding recruitment efforts 
to increase the numbers of part-time adult students. This paper 
presents the findings of a market research study to investigate 
why potential, part-time graduate students were not applying for 
admission. This project demonstrates the role institutional 
researchers can play as internal consultants. 




Research Paper 




Moderator: Kathleen Keenan, Director of Institutional Research, 
Massachusetts College of Art 


Noon -1:15 p.m. 


Business Luncheon in the Prince William Ballroom 
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Monday, November 18 continued 




1:30 - 2:15 p.m. 


Using ASQ+ Data to Define and Expand Your View of Your 


Princeton 


Competitors 


Rebecca Brodigan 


This session will focus on developing categories of competitors 


Associate Dean for Enrollment, 


using US News groupings of colleges to look at wins and losses. 


Information Systems & Planning 
Bentley College 


In addition, actual competitor data will be reviewed and the use 
of the data in long range budget planning will be discussed. 

Research Paper 


1:30 - 2:15 p.m. 


A Sector-Wide Survey of Faculty in Private Higher Education: 


Nassau A/B 
Michael McGuire 


One State's Thoughts on the Nature of Faculty Work and Reward 
Systems 


Director of Research 


This paper summarizes a large-scale survey of the full-time 
faculty at 50 private institutions in Pennsylvania. The results 


Jason Casey 


document perceived skill development and research involvement 


Research Associate 


among undergraduates; the complementarity of research and 
teaching; high levels of faculty teaching activity, including 


Pennsylvania Independent 


individualized instruction; and perceived shortcomings in 


College and University Research 
Center 


instructional resources and faculty reward systems. 




Research Paper 


1:30 - 2:15 p.m. 


Glass Ceilings and Sticky Floors in Academe? Is 


Senior 


There Gender/Race Bias in Rank? 


Lois A. Haignere 


This study uses categorical modeling to examine gender and race 


Director of Research 


bias in rank assignments at 12 SUNY institutions, four 
universities, four colleges and four two-year institutions. At two- 


Bonnie Eisenberg 


year schools there is a pattern suggesting that reaching the 


Researcher 


tenured rank of Associate Professor is a glass ceiling for women. 
The four-year college pattern is diverse with the only discernible 


United University Professions 


pattern being what appears to be a sticky floor for women and 
minorities in the non-tenure track rank of Lecturer. There are 
hurdles at most levels for three of the four universities. But the 
results for one university indicate much less bias than the other 
three, suggesting that bias can be avoided. 




Research Paper 
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Monday, November 18 continued 




1:30 - 2:15 p.m. 


Academic Survival Needs of Disadvantaged College Students 


Colonial/University 






Understanding the learning assistance needs of disadvantaged 


Bob Adebayo 


college students at the point-of-entry is one of the initial steps in 


Director of Institutional Research 


developing successful intervention strategies that can eventually 


and Planning 


improve their retention rates. This study shows that the 
academic survival needs of disadvantaged college students differ 


Ohio State University at Newark 


by age, gender, marital status, educational background and 
number of years since leaving school. 

Research Paper 

Moderator: Jane Zeff, Assoc. Director Planning, Research & 


CANCELLED 


Evaluation, William Paterson College 


2:30 - 3:1 5 p.m. 


Using Cohort Analysis to Evaluate the Impact of a Support 


Princeton 


Program for Minority Students 


Hershel Alexander 


This paper presents spring 1995 findings about retention rates. 


Research & Planning Analyst 


earned credits, and grade point averages for four cohorts of first- 
time college students who participated in ALANA, a voluntary 


Prince George's Community 


program that targets minority students in need of developmental 


College 


work in two or three areas (English, reading, or mathematics). 
Research Paper 


2:30 -3:15 p.m. 


Increasing the Utility of the College Board Standard Validity 


Nassau A/B 


Study: A QBasic Program for Predicting an Applicant's College 
Performance 


Jishen Zhao 




Statistician/Analyst 


To increase the utility of the College Board Standard Validity 
Study, an MS-DOS QBasic program was developed to estimate 


The College of Saint Rose 


an applicant's freshman GPA, his/her chances of earning a certain 
GPA or higher, and the expected utility of a select decision and a 
reject decision. 

Workshare 


2:30 - 3:15 p.m. 


Faculty Educational Goals for Students and the Instruction and 
Evaluation Methods They Employ 


Stuart Rich 


Response to the 1995 HERI Faculty Survey from more than 30 


Director of Institutional Research 


selective private institutions will be examined to determine 
whether there are any meaningful relationships between faculty 


Georgetown University 


members' educational goals for undergraduate students and the 
types of instructional and evaluation methods they employ in 
their undergraduate courses. 


RESCHEDULED ON TUESDAY 




MORNING 


Research Paper 
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Monday, November 18 continued 


2:30 - 3:1 5 p.m. 
Colonial/University 


New Approaches to Collecting Survey Responses 




Three new approaches to collecting and encouraging survey 


Marian Pagano 


response at a university are described and analyzed for statistical 


Associate Provost 


and financial efficiencies. 


Columbia University 


Research Paper 


John Pryor 

Coordinator of Evaluation & 
Research 

Dartmouth College 




2:30-3:15 


Who Does the Community College Serve? One College's 


Ships Room 


Experience 


Eleanor Fujita 


In order to better understand how the community college was 


Director of Academic 


serving the county, a standard index was derived and "service 


Information 


rates" were calculated for the population based on different 
characteristics. The results indicated how different groups within 


City University of New York 


the county were being served differentially. 




Workshare 


3: 1 5 - 3:45 p.m. 


Break in the Nassau Foyer: Visit Vendor Exhibits 


3:45 -5:15 p.m. 


Curriculum Assessment at the Molar Level: Evaluating Course 
Objectives. 


Mark Eckstein 
Director of Assessment & 


Assessing educational objectives is part of the institutional 


Institutional Research 


researcher's workday world. College faculty, conversely, often 
have little formal training in educational evaluation. By presenting 


Bruce Hilyare 


the information in an unusual setting — a four-day multimedia 


Professor of English 


workshop -- an assessment professional presented sound practice 
without arousing the "territorial imperative" of professional 


Mary Jane Heider 

Director of Academic Computing 


teachers. 




Panel 


Genesee Community College 




CANCELLED See related 
session Tuesday, 9:00 - 10:00 
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Monday, November 18 continued 




3:45 - 5:15 p.m. 


Admissions Testing 


Princeton 




3:45 - 4:20 p.m. 


The SAT and Recentering 


Bradley Quin 




Director, Admissions & Enrollment 


This year's college bound cohort (those students who enrolled 


Services 


in college this fall) were the first to have their SAT 1: 


The College Board 


Reasoning Test scores reported on the "new" recentered scale 
since the test was placed on scale in 1941. The College 
Board elected to undertake the necessary renorming on the 
heels of having successfully introduced the New SAT in the 
spring of 1994 and in light of the fact that the reference 
group taking the test today is substantially different from the 
testing population of 1941. 

This session will review the policy, technical, and 
communications issues associated with this change. Various 
reactions to recentering by College Board member institutions, 
the press and media, as well as public policy makers will be 
discussed. 


4:30 - 5:05 p.m. 


Research Paper 


Nancy Burton 




Director Development & Research 


Admissions Testing: Does it Have a Future? 


Educational Testing Service 






In the final years of the twentieth century, admissions tests 
created in the 1920s or even the 1950s seem curiously dated. 

A cafeteria full of students, stacks of test books, number 2 
pencils -- is there a future for this kind of testing? Is there a 
future for multiple-choice tests of general skills, rather than 
measures of very specific performances in realistic settings? 
Will national tests still be used, rather than assessments of 
specific state and district educational standards? These 


NEW SESSION 


issues will be discussed from one test publisher's perspective. 
Research Paper 


3:45 - 5:1 5 p.m. 


Retention Revisited 


Nassau A/B 




3:45 - 4:20 p.m. 


Developing a Profile of Retained and Attrited Students 


Tracy Hunt-White 




Asst. Director Planning and 


Implementing effective retention policies require knowledge of 


Institutional Research 


the academic and non-academic characteristics of returning 
and non-returning students. This study will develop a profile 


The Catholic University of America 


of these students using the following variables: satisfaction 
with college services, demographics, high school background, 
college choipe, academic performance, institutional support, 
and reasons for leaving. 
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Monday, November 18 continued 


Research Paper 


4:30 - 5:05 


Why Students Failed to Return 


Marcia M. Lee 




Director of Institutional Research 


Student success -- whether it be to earn a degree, transfer to 
a four-year college, prepare for a new career, or take courses 


Westchester Community College 


for personal interest - is a primary focus of a community 
college. In spring 1994 a survey was sent to all first-time fall 
1993 freshmen who did had not returned for the spring. The 




results and the method used are the topic of this paper. 




Research Paper 

Moderator: Yves M. Gachette, Assistant for Institutional 
Research, Buffalo State College 


3:45 - 5:1 5 p.m. 


Finding Peers 


Senior 




3:45 - 4:20 p.m. 


In Search of Peer Institutions: Two Methods of Exploring and 


Bruce Szelest 


Determining Peer Institutions 


Associate for Institutional Research 




SUNY-Albany 


This paper details the use of Factor and Cluster Analysis, as 


well as a Composite Variable Rank method, to develop peer 
institution groupings for a university. A broad array of 
variables representing institutional dimensions of finance, size, 
complexity, quality, and graduate education emphasis are 
used. The advantages and limitations of each method are 
discussed, as are issues of data availability and integrity. 

Research Paper 


4:30 - 5:05 p.m. 


Timothy A. Walsh 


Identifying Peer Institutions for Graduation Rate Comparisons 


Director of Student Information 


Systems 


This presentation will use publicly available student measures 
to identify similar institutions for graduation rate comparisons. 


Temple University 


The presenter used factor analysis to identify relevant factors 
associated with graduation and then used cluster analysis of 
1200 four-year colleges to identify 31 groups. 

Research Paper 

Moderator: Keith J. Guerin, Director of Institutional Research 
and Planning, County College of Morris 
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Monday, November 18 continued 




3:45 - 5:15 p.m. 
Colonial/University 


Ratings and Their Meaning 


3:45 - 4:20 


Working with the NRC Data on Graduate Programs in 


Robert Heffernan 


the U.S.: Considerations and Concerns. 


Institutional Research & Planning 


Working with data from the recently published NRC 


Tina Grycenkov 


report provides the institutional researcher with a rich 


Paul Snyder 


repository of data that can be used to influence the 
discussion on the quality of American doctoral 


Rutgers University 


education. Some of the uses and problems, along with 
some of the findings, of working with these data are 
presented. 


4:30 - 5:05 p.m. 

Robert K. Toutkoushian 


Workshare 


Executive Director, Office of Policy 


The NRC Graduate Program Ratings: What Are They 


Analysis 


Measuring? 


University System of New Hampshire 


In this paper, we analyze the graduate program ratings 
derived from the 1993 National Research Council (NRC) 
survey. We show that the ratings are influenced by a 
series of faculty, institutional, and student 
characteristics. We also compare and contrast program 




ratings for scholarly quality and program effectiveness. 
Research Paper 

Moderator: Stephen W. Thorpe, Director of Institutional 
Research, La Salle University 
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Tuesday, November 19 


7:30 - 9:00 a.m. 


Breakfast in the Prince William Ballroom; Concurrent 
At-Your-Service Sessions and Special Interest Groups 


8:00 - 8:50 a.m. 


ASQ and ASQ+ Users Group 


Table 1 


An opportunity for those interested in the Admitted 


Ellen Armstrong Kanarek 


Student Questionnaire, or Admitted Student 


Vice President 


Questionnaire Plus, to discuss their experiences, have 
their questions answered, get help on specific analyses 


Applied Educational Research, Inc. 


of your ASQ/ASQ+ data, and learn what changes may 


******************************* 


be planned. 




Special Interest Group 


8:00 - 8:50 a.m. 
Table 2 


Questionnaire Development: Maximizing Information 


Lynn Rothstein 


Within the Institution, Over Time and Across 


Executive Associate to the President 


Disciplines. 


Union Theological Seminary 


Sharing questionnaire development and content is the 


Mary Jean Whitelaw 


focus of this session. Instruments were developed to 


Carnegie Foundation for the 


maximize information within an institution (students. 


Advancement of Teaching 


faculty, deans, alumni), over time (entering, continuing. 


Peter Syverson 


exiting students, alumni), and across disciplines (i.e.. 


Council for Graduate Schools 


English, mathematics, political science). This material is 


******************************* 


useful to both graduate and undergraduate institutions. 




AYS-Table Topic 


8:00 - 8:50 a.m. 
Table 3 




Jane Zeff 

Assoc. Director of Planning, Research & 
Evaluation 


Expanding the Audience for IR Information. 


William Paterson College 


Examples of several formats will be distributed and 
serve as a basis for discussion about how and why 


Eleanor Fujita 


these reports were produced and some of the reactions 
the reports have received. 




AYS-Table Topic 
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Tuesday, November 19 continued 


7:30 - 9:00 a.m., continued 


Breakfast in the Prince William Ballroom, 
continued; Concurrent At-Your-Service 
Sessions and Special Interest Groups 


8:00 - 8:50 a.m. 


Higher Education Data-Sharing Consortium 


Senior Room 


An opportunity for HEDS members and others 


James F. Trainer 


interested in data exchange activities to 


Director 


discuss current issues and concerns. 


Higher Education Data Sharing Consortium 


Special Interest Group 


************************************** 




8:00 - 8:50 a.m. 
Table 4 


Focus Group 1: The Princeton Conference 


Denise A. Krallman 
Institutional Research Analyst 


A focus group to elicit information on the 
strengths and weaknesses of this year’s 


Miami University of Ohio 


conference, led by the program chair of next 
year's conference. By invitation. 


*»»»»»»»»»»»»»»»»»»»»»»»»»»*********** 


AYS-Table Topic 


8:00 - 8:50 a.m. 
Table 5 




Robert A. Yanckello 


Focus Group II: The Princeton Conference 


Assistant Director of Planning 
Central Connecticut State University 


A focus group designed to elicit information 
on the strengths and weaknesses of this 
year's conference, led by the local 
arrangement's chair for next year's 
conference. By invitation. 




AYS-Table Topic 


9:00 - 10:00 a.m. 


Decision Making in Higher Education: A Many 


Princeton 


Sided Coin 


Charles Border 


A qualitative research study was conducted at 


Coordinator, Institutional Research 


a SUNY college to gather information 
regarding the perceptions of the faculty and 


. D'Youville College 


administration of their role in the decision 
making process. The results show how 


CANCELLED 


important it is for IR practitioners to consider 
more than quantitative techniques when 
approaching a policy question. 


. > 


Research Paper 



Tuesday, November 19 continued 


9:00 - 10:00 a. m. 


Comparative Perspectives on the Role of 


Nassau A/B 


Institutional Research: Variation by 
Institutional Characteristics 


Anne Marie Delaney 
Director of Program Research 


Based on a survey of 243 New England 
colleges and universities, this paper presents 


Boston College 


results from bivariate and multivariate 
analyses showing the relationships among the 
following: institutional characteristics, the 
nature of the institutional research function 
and the kind of research performed, the 
contribution of institutional research to 
decision making, the vision for the role of 
institutional research and the estimated 
resources required to fulfill this vision. 




Research Paper 




Recipient of the "Best Paper" award at the 
Burlington Conference, 1995. 




Moderator: Bob Adebayo, Director of 
Institutional Research and Planning, Ohio 
State University at Newark 


9:00 - 10:00 a.m. 


Student-Centered No Show Study 


Brenda Bailey 


In the 1980s institutional research regularly 


Coordinator of Institutional Research 


surveyed "no show" students and produced 
reports aimed at improved marketing. This 


Edinboro University of Pennsylvania 


current study focusses not on marketing,. but 
on what a university can do to become more 
student-centered and encourage accepted 


CANCELLED 


students to matriculate. Topics suggested by 
parents for improvement will be further 
assessed as part of the continuous 
improvement process. 




Research Paper 


9:00 - 10:00 a.m. 
Senior 


Faculty Educational Goals for Students & 
lnstruction& Evaluation Methods They Employ 


Stuart Rich 

Director of Institutional Research 


Response to the 1995 HERI Faculty Survey 
from more than 30 selective private 
institutions will be examined to determine 


Georgetown University 


whether there are any meaningful relationships 
between faculty members' educational goals 
for undergraduate students and the types of 
instructional and evaluation methods they 
employ in their undergraduate courses. 
Research Paper 
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Tuesday, November 19 continued 


9:00 - 10:00 a.m. 


Curriculum Assessment at the Molecular 


Colonial/University 


Level: Evaluating Course Objectives. 


Mark Eckstein 


Assessing educational objectives is part of the 


Director of Assessment & Institutional Research 


institutional researcher's workday world. 
College faculty, conversely, often have little 


Genesee Community College 


formal training in educational evaluation. By 
presenting the information in an unusual 
setting -- a four-day multimedia workshop - 
an assessment professional presented sound 
practice without arousing the "territorial 
imperative" of professional teachers. 




Workshare 


9:00 - 10:00 a.m. 


Taking Values Seriously: What's All This Data 
Telling Us? 


Bayard Baylis 
Associate Dean 


This multi-year project uses quantitative and 
qualitative means with students, alumni and 


Messiah College 


faculty to determine how well colleges are 
doing in shaping the values of their students. 
This paper presents baseline data and early 


CANCELLED 


comparative results from the first two years of 
the project. 




Research Paper 


10:00 - 10:30 a.m. 


Break in the Nassau Foyer 


10:30 a.m. - Noon 


Thoughts of One's Own: Speculations about 


Ballroom 


What the Future Holds for Higher Education 
and Implications for the Practice of 


Jack Dunn 


Institutional Research 


Terry Hartle 


Moderator: Sandy Johnson, Assistant Dean 


Vice President for Government Relations, American 


of the Faculty, Princeton University 


Council on Education 


Closing Plenary Session 


Richard Spies 
Financial Vice President 
Princeton University 




Noon - 4:30 


Luncheon and Meeting of the Steering 
Committee in Ships 
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Hershel Alexander 

Rsch/Plng Analyst 

Prince George’s Comm Coll 

Kent Hall 230 

301 Largo Rd 

Largo, MD 20774-2199 

301-322-0741 

301-808-0960 

HA20 PGSTUMAIL . PG . CC . MD . US 
P 



Jean Anderson 
Registrar/Dir , IR 
King’s College 
133 N River St 
Wilkes-Barre, PA 18711 
717-826-5870 
717-825-9049 
jpanders0rsO2 . kings . edu 
F 



Kelli Armstrong 
Research Analyst 
Tufts Univ 
28 Sawyer Ave 
Medford, MA 02155 
617-628-5000 X5246 
617-627-3993 

KARMSTRONG0 INFONET . TUFTS . EDU 
P 



Sister Virginia Assumpta 
Director, IR & Planning 
Immaculata College 
P.O. Box 702 

Immaculata, PA 19345-0702 
610-647-4400 X3147 
610-251-1668 
P 



K Tracy Barnes 

Asst Dean, Curricular Rsch 

Brown Univ 

PO Box K 

Providence, RI 02912 

401-863-1914 

401-863-7542 

katharine_barnes0brown . edu 
P 



Catherine Alvord 

Data Analyst 

Cornell Univ 

440 Day Hall 

Ithaca, NY 14853-2801 

607-255-7546 

607-255-4612 

CJA20CORNELL.EDU 

P 



Tommy Annas 
Asst Provost , IR 
SUNY-Central Admin 
Inst Rsch & Anlytic Stds 
State University Plaza 
Albany, NY 12246 
518-443-5639 
518-443-5632 
annast0sysadm. suny.edu 
P 



Wayne S Arndt 
Director, IR 
Georgian Court Coll 
900 Lakewood Ave 
Lakewood, NJ 08701-2697 
908-364-2200 X238 
908-367-3920 
warndt0georgian . edu 
F 



Lawrence Baldwin 
Director, Operations 
Wellesley College 
106 Central St 
Wellesley, MA 02181 
617-283-3183 
617-283-3663 
LBALDWIN0WELLESLEY.EDU 
P 



Karen Bauer 

Asst Director, Instl R&P 
Univ of Delaware 
325 Hullihen Hall 
Newark, DE 19716 
302-831-2021 
302-831-8530 

KAREN . BAUER0MVS . UDEL . EDU 
P 
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Connie Beale 

Dir, Quant Analysis 

Seton Hall Univ 

Planning Office 

South Orange, NJ 07079 

201-761-9401 

201-761-9793 

BEALECON0LANMAI L . SHU . EDU 
F 



Susan Bello 

Instl Rsch 

Nassau CC 

1 Education Dr 

Garden City, NY 11530 

516-572-8160 

M 



Craig S. Billie 
Assoc, for IR 
SUNY-Central Admin 
Design 

State University Plaza 
Room 309 

Albany, NY 12246 

518-443-5639 

518-443-5632 

billiecs0sysadm. suny . edu 
P 



Peter Blanchfield 
Asst Director, IR 
Hamilton College 
198 College Hill Rd 
Clinton, NY 13323 
315-859-4045 
315-859-4185 
PBLANCHF0HAMILTON.EDU 
P 



Gary Blose 
Associate for IR 
SUNY-Central Admin 
State University Plaza 
Albany, NY 12246 
518-443-5639 
518-443-5632 
blosegl0sysadm . suny. edu 
V097 J3E50UBVMS . CC . BUFFALO.EDU 
P 



Cheryl Beil 

Dir, Enrlmt Rsch/Ret 

George Washington Univ 

Suite 102 

2121 Eye St, NW 

Washington, DC 20052 

202-994-6712 

202-994-0357 

cbeil0gwl52 . circ . gwu : edu 
P 



Elizabeth Be r aha 

Programmer /Analyst 

Boston Univ 

2nd FI, Rm 244 

881 Commonwealth Ave 

Boston, MA 02215 

617-353-9094 

617-353-7300 

erlejb0uism.bu.edu 

M 



Felice D. Billups 

Director, IR 

Rhode Island School of 

2 College St. 
Providence, RI 02903 
401-454-6334 
401-454-6406 
M 



Marilyn H. Blaustein 

Director, IR 

UMASS -Amherst 

237 Whitmore Building 

Box 38190 

Amherst, MA 01002 

413-545-0941 

413-545-3010 

BLAUSTEIN0OIRP. UMASS . EDU 
P 



Charles Border 
Coordinator, IR 
D*Youville College 
320 Porter Ave 
Buffalo, NY 14201 
716-881-7727 
716-881-7790 
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Karl Boughan 

Supervisor, IR 

Prince George’s Comm Coll 

301 Largo Rd K-231 

Largo, MD 20774 

301-322-0722 

301-808-0960 

KB2 0 PGSTUMAIL . PG . CC . MD . US 
P 



Kathleen Kern Bowman 
Research Associate 
COFHE 

238 Main Street, Suite 307 

Cambridge, MA 02142 

617-253-5032 

617-258-8280 

KKERN0MIT.EDU 

P 



David Breen 

VP Plng/Admin 

Merrimack College 

315 Turnpike St 

North Andover, MA 01845 

5:08-837-5231 

508-837-5222 

DBREEN0MERRIMACK . EDU 

M 



Rebecca Brodigan 
Director, Plng/Rsch 
Vermont State Colleges 
PO Box 359 

Waterbury, VT 05676-0359 
802-241-2520 
802-241-3369 
brodiganr0quark . vsc . edu 
P 



Jennifer A. Brown 

Exec Officer, Ac Aff/Rsch 

Connecticut State Univ System 

39 Woodland St 

Hartford, CT 06105-2337 

860-493-0074 

860-493-0080 

BROWN J0CSUSYS . CTSTATEU . EDU 
P 



Nancy Burton 

Director, Rsch&Dev 

Educational Testing Service 

Carter Road 

Princeton, NJ 08541 

609-683-2288 

609-683-2280 

nburton0ets . org 

S 



Brenda Bretz 
Assistant Registrar 
Dickinson College 
P 0 Box 1773 
Carlisle PA 17013 
717-245-1316 
717-245-1534 
BRETZ0DICKINSON.EDU 
P 



C Anthony Broh 
Registrar 
Princeton Univ 
101 West College 
Princeton, NJ 08502 
609-258-6191 
609-258-6328 
broh0princeton.edu 
F 



0. Richard Bundy, III 

Dir., Telefund Operations 

Penn State 

112 S Burrowes St 

State College, PA 16801 

814-863-3935 

orblOO0psu . edu 

F 



Kathleen M Busch 

Asst to President 

Assoc of Ind Col & Uniy NJ 

797 Springfield Ave 

Summit, NJ 07901 

908-277-3738 

908-277-0851 

MAILO77130POP.NET 

M 
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Rita Catalano 

Director, Grants & Research 
St Vincent College of PA 
300 Fraser Purchase Rd 
Latrobe, PA 15650-2690 
412-537-4562 
412-537-4587 
RCATALANOeSTVINCENT . EDU 
F 



Rena Che skis -Gold 

Sr Rsch Analyst 

Yale Univ 

246 Church St 

PO Box 208289 

New Haven, CT 06520-8289 

203-432-1331 

203-432-1754 

RENA . CHESKIS-GOLD@YALE . EDU 
P 



Salvatore Ciolino 
Director, Instl Analysis 
Gettysburg College 
325 Table Rock Rd 
Gettysburg, PA 17325 
717-337-6582 
717-337-6008 
SCIOLINO0GETTYSBURG . EDU 
P 



Margaret K. Cohen 
Asst VP for Instl Rsch 
George Washington Univ 
Rice Hall, Suite 809 
Washington, DC 20052 
202-994-6509 
202-994-0709 
SISMKC0GWUNIX2 . GWU . EDU 
P 



Corby A Coperthwaite 
Director, IR 

Manchester Comm-Tech College 
60 Bidwell St 
PO Box 1046 

Manchester, CT 06045-1046 
860-647-6101 
860-647-6332 
MA_CORBY0COMMNET . EDU 
P 



Paula Cox 

Sr Rsch Officer/IR 
Rice Univ 
Suite 200, MS95 
5620 Greenbriar 
Houston, TX 77005 
713-831-4700 
713-831-4747 
pncox0rice . edu 
F 



Craig A. Clagett 
Director, IR & Analysis 
Prince George’s Comm Coll 
301 Largo Road, K-231 
Largo, MD 20774 
301-322-0723 
301-808-0960 

CC50 PGSTUMAIL : PG . CC . MD . US 
P 



Fred Cohen 

Dir, Enrlmt Rsch/Analysis 
New York Univ 
Suite 615 

7 East 12th Street 

New York, NY 10003-4475 

212-998-4415 

212-995-4095 

FRED . COHEN0NYU . EDU 

P 



Mary Ann Coughlin 
Professor, Rsch/Stats 
Springfield College 
Alden St 

Springfield, MA 01109 

413-748-3038 

413-748-3537 

mary_coughlin0spf Idcol . edu 
P 



Diane Cuneo 
Director, IR 
Smith College 
Office of Instl Res 
Northampton, MA 01063 
413-585-3022 
413-585-3026 
DCUNEO0SMITH . SMITH . EDU 
P 
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Cherry Danielson 
Graduate Rsch Asst 
Univ of Michigan 
582 South Seventh St 
Ann Arbor, MI 48103 
313-747-8849 
CHERRYD0UMICH . EDU 
P 



David L. Davis-Van Atta 

Director, IR 

Carleton College 

One N College St 

Northfield, MN 55057 

507-646-4292 

507-646-4540 

DDAVISVA0ACS . CARLETON . EDU 
P 



Karen DeMonte 
Research Analyst 
Univ of Delaware 
Office of IR 
325 Hullihen Hall 
Newark, DE 19716 
302-831-2021 
302-831-8530 

karen . demonte0mvs . udel . edu 
F 



Michael Dooris 
Dir, Acad Assessment 
Penn State Univ 
417 Old Main 

University Park, PA 16802 

814-863-1864 

814-863-7452 

mjdl0psu . edu 

P 



John A. Dunn, Jr. 

Executive Director 

Mt Washington Observatory 

PO Box 294 

Jackson, NH 03846 

603-383-6615 

603-383-6615 

JDUNN0TIAC.NET 

P 



Arlene Danville 
Assoc Director, IR 
Pace Univ 
1 Pace Plaza 
New York, NY 10038 
212-346-1429 
212-346-1041 
danville0ny2 . pace . edu 
F 



Anne Marie DeLaney 
Director, Program Research 
Boston College 
Campion Hall 336A 
Chestnut Hill, MA 02167 
617-552-0682 
617-552-8419 
DELANEYA0HERMES . BC . EDU 
P 



Kimberley Dolphin 
Coordinator, IR 
Bentley College 
175 Forest St 
Waltham, MA 02154 
617-891-2551 
617-891-2569 
KDOLPHIN0BENTLEY.EDU 
P 



Scott Drakulich 

Director, IR 

Essex Co Coll 

303 University Ave 

Newark, NJ 07737 

201-877-3370 

201-623-6449 

DRAKULI0PILOT . NJIN . NET 

F 



Jeffrey Dutton 
Assistant Provost 
SUNY-Buf falo 
406 Capen Hall 
Box 601603 

Buffalo, NY 14260-1603 

716-645-2791 

716-645-3799 

DUTTON0MSMAIL . BUFFALO . EDU 
P 




Mark A. Eckstein 
Dir, Assess & IR 
Genesee Comm Coll 
One College Road 
Batavia, NY 14020 
716-343-0055 X6549 
716-343-4541 

markeOsgccvb . sunygenesee . cc . ny . us 
P 



Rhoda Epstein 
Director, IR 
Sullivan Co CC . 

1000 Leroy .Rd 

Loch Sheldrake, NY 12759 

914-434-5750 X221 

914-434-4806 

F 



Barbara Erdsneker 
Asst Director, IR 
Bergen Comm Coll 
400 Paramus Road 
Paramus, NJ 07652 
201-447-9207 
201-445-8532 
erdsnek0 email .njin.net 
P 



Ayshe Ergin 
Director, IR 

The College of New Jersey 
Green Hall 101 
Hillwood Lakes, CN 4700 
Trenton, NJ 08650-4700 
609-771-2797 
609-771-3484 
erginaOtcnj .edu 
F 



Marilyn Feingold 

Asst Dean, IR 

Camden County College 

PO Box 200 

Blackwood, NJ 08012 

609-374-4932 

609-374-4892 

FeingoldOpilot .njin.net 

P 



James C. Fergerson 
Director, IR 
Bates College 
2 Andrews Road 
Lewiston, ME 04240-6028 
207-786-8210 
207-786-6123 

JFERGERS0ABACUS . BATES . EDU 
P 



James Firnberg 
President 

Our Lady of the Lake College 

5345 Brittany Dr 

Baton Rouge, LA 70808 

504-768-1710 

504-768-1726 

JFIRNBERG0OLOLCOLLEGE . CCLA . US 
F 



Phyllis A. Fitzpatrick 
Director, Mgmt Info 
Fairfield Univ 
1073 N Benson Rd 
Fairfield, CT 06430-5195 
203-254-4000 x2774 
203-254-4007 
PFITZPATRICK0FAIR1 . 

FAIRFIELD.EDU 

P 



Thomas Frank 

Director, IR 

Providence College 

River Ave & Eaton St 

Providence, RI 02918-0001 

401-865-2723 

401-865-2236 

t f rank0providence . edu 

F 



Nurit Friedman 

Data Services Manager 

Villanova Univ 

800 Lancaster Ave 

Villanova, PA 19085 

610-519-7320 

610-519-7162 

NFRIEDMA0EMAIL.VILL.EDU 

P 
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Karen Froslid 

Asst Dir, Univ Plng/Rsch 

American Univ 

4900 Massachusetts Ave, NW 

Washington, DC 20016 

202-244-8180 

202-885-3273 

KFROSLI 0AMERI CAN . EDU 

F 



Rhonda Gabovitch 
Director, IR 
Massasoit CC 
1 Massasoit Blvd 
Brockton, MA 02402 
508-588-9100 X1705 
508-427-1205 

RGABOVITCH0MECN . MASS . EDU 
P 



Gary Gaffield 
Asst Prov, Ac Prog 
Wittenberg Univ 
PO Box 720 

Springfield, OH 45501 

513-327-7922 

513-327-6340 

gaf fie ld0 Wittenberg . edu 

F 



■ Lorraine Gill 
Asst to Pres, IR 
Eastern Nazarene College 
23 E Elm Ave 
Quincy, MA 02170 
617-745-3486 
617-745-3490 
gilll0enc.edu 
M 



Jerene Good 

Director, IR 

Drexel Univ 

3141 Chestnut St 

Philadelphia, PA 19104-2875 

215-895-1843 

215-895-1508 

JERENE0DUVM. OCS . DREXEL . EDU 
P 



Eleanor Fujita 
Director, Acad Info ■ 
City Univ of NY 
155 Calhoun Ave . 

New Rochelle NY 10801 
212-794-5556 
212-794-5706 
elfbh0cunyvm . cuny . edu 
P 



Yves M. Gachette 
Asst for IR 
Buffalo State College 
1300 Elmwood Ave 
Buffalo, NY 14222 
716-878-4521 - 
716-878-3140 
F 



Janice A Gifford 
Assoc Dean of Faculty 
Mount Holyoke College 
102 Mary Lyon Hall 
South Hadley, MA 01075 
413-538-2874 
413-538-2391 
j gif ford0mtholyoke . edu 
F 



Ivan Gonzalez 

Senior Analyst 

Columbia Univ 

211 Low Memorial Library 

New York, NY 10027 

212-854-3073 

212-854-9493 

igi 70 Columbia . edu 

P 



Indira Govindan 
Director, IR 
Drew Univ 
36 Madison Ave 
Madison, NJ 07940 
201-408-3636 
201-408-3038 
IGOVINDA0DREW.EDU 
P 
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Helen Gradisar 
Director, IR 
Carlow College 
3333 Fifth Avenue 
Pittsburgh, PA 15213 
412-578-6077 
412-578-8710 
M 



Rosalinda Graham 
Sr . Research Analyst 
Univ of Delaware 
334 Hullihen Hall 
Newark, DE 19711 
302-831-6283 
302-831-8530 
ROSALINDA . GRAHAM0 
MMVS.UDEL.EDU 
P 



Richard Grakowsky 
Technical Asst 
Monroe CC 

1000 East Henrietta Rd 
Rochester, NY 14623-5780 
716-292-3033 
716-427-2749 

GRAKOWSK0MCCVM .IS. MONROECC . EDU 
F 



Joseph Grasso 
Dir, Budget /I R/ Ping 
Colgate Univ 
13 Oak Dr 

Hamilton, NY 13346 

315-824-7455 

315-824-7861 

JGRASSO0CENTER . COLGATE . EDU 
F 



Martha Gray 

Director, IR 

Ithaca College 

340 Job Hall 

Ithaca, NY 14850-7010 

607-274-3164 

607-274-1500 

mgray0ithaca . edu 

P 



Jean Gubbins 

Program Mgr, Public Affairs 
Case Western Reserve Univ 
10900 Euclid Ave 
Cleveland, OH 44106-7021 
216-368-5557 
216-368-6674 
JEG20PO.CWRU.EDU 
M 



Heather Haber aecker 
VP Finance & Admin 
Clarion Univ of PA 
218 Carrier Hall 
Clarion, PA 16214 
814-226-2240 
814-226-1826 
0 



Tina Grycenkov 

Info Mgr 

Rutgers Univ 

Geology Hall, 1st FI 

New Brunswick, NJ 08903 

908-932-7305 

908-932-1268 

GRYCENKOV0INSTLRES . RUTGERS . EDU 
P 



Keith J. Guerin 
Director, IR/Plng 
County College of Morris 
214 Center Grove Rd 
Randolph, NJ 07869 
201-328-5056 
201-328-5053 
KGUERIN0CCM.EDU 
P 



Lois Haignere 
Director of Research 
United University 
Professions 
159 Wolf Rd 
Albany, NY 12205 
518-458-7935 
518-459-3242 
HAIGNER0ALBANY . NET 
P 
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Michael Haines 
Director, IR/Plng/Assmt 
Dominican College 
470 Western Hwy 
Orangeburg, NY 10962 
914-359-7800 x359 
914-359-2313 
MH1250COLUMBIA.EDU 
P 



Martha Hayden 
Senior Programmer Analyst 
Brown Univ 
Box K 

Providence, RI 02912 
401-863-2386 
401-863-7542 
martha_hayden0brown . edu 
F 



Robert J. Heffernan 
Assoc Dir, IR 
Rutgers Univ 
Office of Ac Ping & IR 
Geology Hall, 1st FI 
New Brunswick, NJ 08903 
908-932-7305 
908-932-1268 

HEFFERNAN0INSTLRES . RUTGERS . EDU 
P 



David Hemenway 
Director, Ping & IR 
Eastern CT State Univ 
83 Windham St 
Willimantic, CT 06226 

860-465-5306 

860-465-4485 

hemenwayd0ecsuc . ctstateu . edu 
P 



Gail Hermosilla 
Director, Plng/IR 
Dutchess CC 
53 Pendell Road 
Poughkeepsie, NY 12601 
914-431-8680 
914-431-8984 

hermosil0SUNYDutchess . edu 
M 



Gary Hammon 

Univ of Massachusetts 

150 Whitmore Bldg 

Amherst, MA 01075 

413-545-3995 

413-545-3205 

G . HAMMON0DPC . UMASSP . EDU 

P 



Richard C. Heck 
Ex Officer/Dean of College' 
Dartmouth College ■ 

Box 6003 

Hanover, NH 03755 
603-646-3800 
603-646-1405 
R.HECK0DARTMOUTH. EDU 
P 



Michael H Heiberger 

Director, Plng/Eval 

SUNY-College of Optometry 

100 E 24th St 

New York, NY 10010 • 

212-780-509T 

212-780-4949 

heiberg0 sunyopt . edu 

F 



Ann Henderson 
Registrar & Dir, IR 
Skidmore College 
815 North' Broadway 
Saratoga Springs, NY 
12866-1632 
518-584-5000 X2210 
518-584-7963 
AHENDERS0SKIDMORE . EDU 
P 



Jeffery J. Himmelberger 
Admissions Rsch Analyst 
Clark Univ 
950 Main St 
Worcester, MA 01610 
508-793-7374 
508-793-7500 

JHIMMELBERGE0VAX . CLARKU . EDU 
P 
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Larene Hoelcle 
Dean, Human Res/Plng 
Genesee Comm Coll 
One College Road 
Batavia, NY 14020 
716-345-6811 
716-343-4541 

LARENEH0SGCCVB . SUNYGENESEE . CC . NY . US 
M 



Kimberly Hoeritz 

Dir., Rsch for Higher Ed 

Peterson * s 

202 Carnegie Center 

Princeton, NJ 08543-2123 

V 



Marie Huester 
Asst Director, IR 
Marywood College 
2300 Adams Ave 
Scranton, PA 18509-1598 
717-348-6203 
717-348-1899 
huesterOac . marywood . edu 
P 



Sherrill Ison 
Director, IR 
Monroe Comm Coll 
1000 E. Henrietta Rd. 
Rochester, NY 14623 
716-292-3032 
716-427-2749 

isonOmccvm. is .monroecc. edu 
P 



Tracy Hunt-White 

Assoc Director, Plng/IR 

Catholic Univ of America 

Cardinal Station PO 

Washington, DC 20064 

202-319-6425 

202-319-6690 

HUNTT0CUA.EDU 

P 



Sally Jack 
Assoc for IR 
SUNY-Central Admin 
State University Plaza 
Albany, NY 12246 
518-443-5639 
518-443-5632 
JACKSE0SNYCENVM 
P 



Robert Jalbert 

Director of Research 

Norwalk Community-Tech College 

188 Richards Avenue 

Norwalk, CT 06854 

203-857-7081 

203-857-7297 

NK_JALBERT0COMMNET . EDU 

P 



Jennifer Jilek 
Rsch Statistician 
Univ of Pittsburgh 
2nd FI, Bruce Hall 
Pittsburgh, PA 15260 
412-648-7148 
412-648-8815 
jilek0oafa.pitt.edu 
M 



Sandra Johnson 

Asst Dean of the Faculty 

Princeton Univ 

Room 304 Nassau Hall 

Princeton, NJ 08544-5264 

609-258-5230 

609-258-2518 

SANDY J0 PRINCETON . EDU 

P 



Tom Jones 
President 

Scanning Products, Inc 
11402 Gayton Rd 
Richmond, VA 23233 
V 
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Ellen Armstrong Kanarek 
Vice President 

Applied Educational Research, 
100 Thanet Circle 
Princeton, NJ 08540 
609-924-0464 
609-924-2025 

KANAREK0 PUCC . PRINCETON . EDU 
P 



Kathleen A. Keenan 
Director, IR 

Inc. Mass. College of Art 

621 Huntington Ave 
Boston, MA 02115 
617-232-1555 X242 
617-232-0050 
KKEENAN0MECN . MASS . EDU 
P 



Robert L Kegebein III 
Rsch Analyst 
Corning CC 
1 Academic Dr 
Corning, NY 14830 
607-962-9211 
607-962-9246 

kegebein0scccvc . corning- cc . edu 
F 



Gurvinder K Khaneja 

Research Asst 

Union Co Coll 

1033 Springfield Ave 

Cranford, NJ 07016-1599 

908-709-7508 

908-709-0827 

KHANEJA0HAWK . UCC . EDU 

F 



Yun K. Kim 

Director, IR 

Charles Co CC 

PO Box 910 

La Plata, MD 20646 

301-934-7620 

301-934-7679 

yunk0 Charles . cc . md . us 

P 



John Kelley 

Exec Dir, Plng/IR 

Villanova Univ 

800 Lancaster Ave 

Villanova, PA 19085-1699 

610-519-4835 

610-519-7162 

kelley- j 0 email .vill.edu 

P 



Robyn Dickinson -Kiefer 

Director, IR 

Lehigh Carbon Comm Coll 

4525 Education Park Dr 

Schnecksville, PA 18078 

610-799-1592 

610-799-1187 

ROBYNDK0AOL.COM 

P 



Richard L. Kline 
Director, IR 
California Univ of PA 
250 University Ave 
California, PA 15419-1394 
412-938-4265 
412-938-4265 
KLINE0CUP.EDU 
P 



Denise A. Krallman 
Instl Rsch Analyst 
Miami Univ-Oxford 
Budgeting/ Planning/Analysis 

213 Roudebush Hall 
Oxford, OH 45056 
513-529-7095 
513-529-2121 
krallmda0muohio . edu 
P 



Arthur Kramer 

Director, IR 

Jersey City State Coll 

2039 Kennedy Blvd 

Jersey City, NJ 07305-1597 

201-200-2499 

201-200-3141 

akramer0 jcsl . jest ate . edu 
P 
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John Kraus 

Director, IR 

Univ of New Hampshire 

20 Rosemary Lane 

Palmer House 

Durham, NH 03824-3537 

603-862-2410 

603-862-3853 

J_KRAUS0UNHN . UNH . EDU 

P 



Linda Lam 

Policy Rsch Officer 
NJ Council Co Coll 
330 West State St 
Trenton, NJ 08618 
609-392-3434 
609-392-8158 
LAMLINDA0AOL.COM 
F 



David Lasky 

Director, IR 

Lebanon Valley College 

Annville, PA 17003 

717-867-6204 

717-867-6075 

LASKY0LVC.EDU 

P 



Linda LeFauve 
Director, IR 
Middlebury College 
Middlebury, VT 05753 
802-443-5871 
802-443-2067 

lef auve0msmail .buffalo . edu 
P 



Alberta Lipson 

Assoc Dean for Research 

MIT 

18 Vassar St 20B-140 

Cambridge, MA 02139 

617-253-8604 

617-253-0055 

LIPSON0MIT.EDU 

P 



Jeffrey Kuenzi 
Research Analyst 
Smith College 
207 Clark Hall 
Northampton, MA 01063 
413-585-3045 
j kuenzi 0ais . smith . edu 
F 



Robert D Larsson 

Consultant 

2492 McGovern Dr 

Niskayuna, NY 12309-2434 

518-393-0332 

M 



Marcia M. Lee 
Director, IR/Plng 
Westchester Comm Coll 
75 Grasslands Road 
Valhalla, NY 10595 
914-785-6589 
914-785-6565 
P 



John Lewis 
Research Associate 
Georgetown Univ 
303 Maguire Hall 
Washington, DC 20057 
202-687-3416 
202-687-3935 

LEWIS0GUNET . GEORGETOWN . EDU 
P 



Ruth Loescher 
Research Asst 
Harvard Univ 
453 Holyoke Ctr 
Cambridge, MA 02140 
617-496-3568 
617-495-2900 

ruth_loescher0harvard. edu 
F 
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Wendell G. Lorang 
Assoc. Dir. IR 
SUNY-Albany 
Admin 242 

1400 Washington Ave 
Albany, NY 12222 
518-442-5412 
518-442-5418 
WGL550CNSIBM . ALBANY . EDU 
P 


Nancy Ludwig 
Asst Director, IR 
Northeastern Univ 
360 Huntington Ave 
401 Church Hall 
Boston, MA 02115 
617-373-5100 
617-373-5106 
NLUDWIG0NEU.EDU 
P 


Qing Lin Mack 
Asst for IR 

Connecticut State Univ System 

39 Woodland St 

Hartford, Ct 06105-2337 

860-493-0075 

860-493-0080 

mackqOcsusys . ctstateu . edu 
P 


Joel F. Mack 
Director, Info Tech 
Saint Joseph College 
1678 Asylum St 
West Hartford, CT 06117 
860-232-4571 X244 
860-233-5695 
JMACK0MERCY . S JC . EDU 
M 


Kenneth G. MacKenzie 

Assoc. Dir. Enrl/Plan/Retention 

Boston Univ 

2nd FI, Rm 244 

881 Commonwealth Ave 

Boston, MA 02215 

617-353-4177 

617-353-7300 

kmackenzOuixm. bu . edu 

M 


Shaukat M Malik 
Graduate Student 
SUNY-Albany 
Admin 241 

1400 Washington Ave 
Albany, NY 12222 
F 


Michael Matier 

Assoc Dir, IR/Plng/Rsch 

Cornell Univ 

440 Day Hall 

Ithaca, NY 14853-2801 

607-255-7009 

607-255-4612 

mwm50cornell . edu 

M 


Marie Paulette Matis 
Staff Asst, IR 
Rutgers Univ 
Geology Hall 
1st FI 

New Brunswick, NJ 08903 

908-932-7305 

908-932-1268 

MATIS0INSTLRES . RUTGERS . EDU 
F 


John W. McDermott 

Vice Pres for Ping & Enrollment 

Moravian College 

1200 Main Street 

Bethlehem, PA 18018-6650 

610-861-1555 

610-861-3919 

MEJWMO10MORAVIAN.EDU 

P 


Michael McGuire 

Director of Research 

PICURC 

Suite 404 

800 N Third St 

Harrisburg, PA 17102 

717-232-8649 

717-233-8574 

mcguire0picurc . org 

P 



Catherine McKenna 

Asst VP, Plng/Tech Serv 

Holy Family College 

Grant & Frankford Ave 

Philadelphia, PA 19114-2094 

215-637-7700 X407 

215-637-7700 

MCKENNA0HSLC.ORG 

M 



Billie Miles 

Assoc Dir, Info Tech 

Saint Michael's College 

Winooskie Park 

Colchester, VT 05439 

802-654-2930 

802-654-2422 

BMILES0SMCVT.EDU 

F 



Kathleen M. Morley 

Admin Intern 

UMASS-i\mherst 

237 Whitemore Admin Bldg 

Amherst, MA 01003 

413-545-5145 

413-545-3010 

KMORLEY0SOC . UMASS . EDU 

M 



Ray Mullins 

Sr. Systems Analyst 

Boston Univ 

2nd FI, Rm 244 

881 Commonwealth Ave 

Boston, MA 02215 

617-353-5170 

617-353-7300 

erlrrm0bu . edu 

M 



Kathleen Nazar 
Sr Rsch Analyst 
Villanova Univ 
800 Lancaster Ave 
Villanova, PA 19085-1699 
610-519-6976 
610-519-7162 
P 



Larry W. Metzger 
Dean, Enrollment Ping 
Ithaca College 
340 Job Hall 
Ithaca, NY 14850-7010 
607-274-1555 
607-274-1500 
METZGER0OA. ITHACA.EDU 
P 



Aghajan Mohammadi 
Director, IR 
Capital Comm-Tech Coll 
Asnuntack Comm-Tech Coll 
401 Flatbush Ave 
Hartford, CT 06106-3798 
860-987-4882 
860-987-4822 
mohammadi0commnet . edu 
F 



Joseph Morreale 

Vice Provost Plng/Assmt/IR 

Pace Univ 

1 Pace Plaza, Rm T510 
New York, NY 10038 
212-346-1010 
212-346-1041 

morreale0pacevm .dac.pace.edu 
F 



Peter J . Murray 
Chief Info Officer 
Catholic Univ of America 
Cardinal Station PO 
Washington, DC 20064 
202-319-5373 
202-319-6690 
MURRAYP0CUA. EDU 
P 



Randy Nehila 

Special Studies Analyst 

Boston Univ 

2nd FI, Rm 244 

881 Commonwealth Ave 

Boston, MA 02215 

617-353-4113 

617-353-7300 

er IrsnObu . edu 

M 
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Thomas Nelson 

Assst Dean/Acad Affairs 

Schenectady Co CC 

78 Washington Ave 

Schenectady, NY 12305 

518-381-1374 

518-346-7511 

nelsont j 0gw . sunysccc . edu 
P 



Elizabeth B. Nolan 
Director, Instl Plng/Rsch 
Siena College 
515 Loudon Road 
Loudonville, NY 12211-1462 
518-783-2380 
518-783-4293 
ENOLAN0SIENA.EDU 
P 



Helene Nemeth 
Asst Director 
William Paterson Coll 
358 Hamburg Turnpike 
PO Box 913 

Wayne, NJ 07470-0913 

201-595-2378 

201-595-3598 

nemethh0gw . wilpaterson . edu 
F 



Mike Novakowski 
SCANTRON 

5793 Wide Waters Parkway 
Dewitt, NY 13214 
V 



Patricia M O'Brien 
Director, IR 
Bridgewater State Coll 
Boyden Hall 
Bridgewater, MA 02325 
508-697-1295 
508-279-6106 
pobrien0bridgew . edu 
F 



Marian F. Pagano 

Associate Provost 

Columbia Univ 

211 Low Memorial Library 

New York, NY 10025 

212-854-2473 

212-854-9493 

MFP30COLUMBIA. EDU 

P 



Carl H Ostermann 

Rsch 'Analyst 

Univ of Pennsylvania 

Box 3, College Hall 

Philadelphia, PA 19104 

215-573-5842 

215-573-5839 

ostermanOdolphin . upenn ! edu 
F 



Barbara H. Palmer 
Director, IR 
Brandeis Univ 
P O Box 9110 
Waltham, MA 02254-9110 
617-736-2010 
617-736-3485 
bpalmerObinah . 

CC . brandeis . edu 
P 



Christopher Peters 
Assoc Director 
Villanova Univ 
800 Lancaster Ave 
Villanova, PA 19085-1699 
610-519-7322 
610-519-7162 
peters-c0email . vill . edu 
P 



Joseph Pettit 
VP, Plng/IR 
Georgetown Univ 
303 McGuire Hall 
Washington, DC 20057 
202-687-3587 
202-687-3935 

PETTIT0GUNET . GEORGETOWN . EDU 
P 
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Diane Phillips 
Director, IR 

Fashion Institute Technology 

227 W 27th St 

New York, NY 10001 

212-760-7178 

212-760-7198 

F 



Richard W. Prull 
Director, IR 
Rhode Island College 
600 Mt . Pleasant Avenue 
Providence, RI 02828 
401-456-8435 
401-456-8209 
RPRULL0GROG . RIC . EDU 
P 



Brad Quin 

Director, Adm & Eval Serv 
The College Board 
45 Columbus Ave 
New York, NY 10023 
V 



Leslie Reggero 
Rsch Assoc 
Mt St Mary Coll 
330 Powell Ave 
Newburgh, NY 12550 

914-569-3192 

914-562-6762 

REGGERO0MSMC.EDU 

F 



Joseph Revelt 
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